Capturing developer interactions with AI tools is the foundational step for observability, cost control and security in modern engineering organizations. This guide shows a practical, tool-agnostic approach to record AI prompts, CLI executions, file diffs and browser assistant activities without changing application source code.
- Capture AI interactions from inside the developer environment to avoid blindspots in headers-only proxies.
- Record pre/post prompts, stdout/stderr, exit codes and structured git diffs for useful audit and replay.
- Use a local-hook + proxy + browser-extension model to create a unified, searchable timeline without touching app code.
- Address privacy with redaction, sampling and expiring links; validate installation with simple test sessions.
Why capture developer AI interactions? The problem statement
Engineering teams now use a mix of CLI agents, local API proxies and browser assistants. Conventional HTTP proxies or analytics that look only at outbound headers leave three blindspots:
- Terminal executions: shell stdout/stderr, tool exit codes and local environment context are not visible to header-only proxies.
- File mutations: plain loggers capture strings but cannot reconstruct structured line-by-line git diffs that show what code actually changed.
- Multi-tool fragmentation: activities scatter across IDEs, CLIs and browser sessions, creating unindexed silos.
Without comprehensive capture you cannot reliably audit behavior, attribute cost to teams, or detect insecure data exfiltration and model-mismatch waste.
Recommended approach: tool-agnostic local capture
Design goals for a practical capture system:
- Zero or minimal friction for developers (no source-code changes).
- Tool-agnostic support across CLI agents, local proxies and web assistants.
- Structured artifacts: prompts, tool inputs, stdout/stderr, exit codes, and file diffs tied to git commits.
- Normalized, provider-neutral timeline for search and analysis.
One implementation pattern that meets these goals is a local, hook-based capture agent plus optional browser extension and proxy. The capture paths combine into a unified timeline index.
Implementation workflow — step-by-step
- Install native hooks
Run two terminal commands to install local hooks that attach to CLI agent runtimes. These hooks intercept pre-execution prompts and post-execution outputs without instrumenting application code.
- Enable local API proxy capture
Optionally route local scripts and LiteLLM-style proxies through a lightweight proxy that records requests/responses while preserving developer workflow.
- Install a browser extension
Capture web-based assistant prompts and replies so that browser interactions are time-aligned with CLI sessions.
- File diff & shell logging
At post-execution, capture structured git diffs for changed files and record stdout/stderr and tool exit codes. Store diffs as line-by-line JSON or a similar structured format to enable replay and search.
- Historical backfill
Run a discovery/backfill tool to surface prior sessions that predate agent installation so the timeline is not blank on day one.
- Normalize into unified timeline
Index events into a provider-neutral schema (timestamp, actor, tool, prompt, inputs, outputs, file diffs, execution metadata) for search and analytics.
Exact artifacts to capture and why they matter
- Pre-execution prompt and parameters: required for attribution and reproducing intent.
- Post-execution output and logs: stdout, stderr and exit codes show what actually ran and whether it succeeded.
- Structured git diffs: line-by-line changes let reviewers and auditors see real edits rather than raw natural language suggestions.
- Provider-neutral metadata: tool name, version, local runtime environment variables and timestamp for tracing incidents and cost allocation.
Validation: how to confirm capture works
- Trigger a simple CLI assistant session that edits a file. Verify the timeline shows the pre-execution prompt, the executed command, stdout/stderr and the structured git diff for the edited file.
- Run a local script through the proxy and confirm requests/responses appear with full bodies (subject to privacy policies).
- Open a browser assistant and perform a query; verify the browser extension recorded prompt and response and that timestamps align with any simultaneous CLI activity.
- Use the historical backfill tool and check that older commits or session artifacts are discoverable in search results.
Edge cases and practical trade-offs
Capture systems face several operational and privacy trade-offs:
- Sensitive data capture: capturing prompts and file diffs risks recording secrets. Implement redaction rules, sampling, and expiring links for reviewers to reduce exposure.
- Performance and developer experience: local hooks must be low-overhead. Prefer asynchronous upload and batching of events to avoid slowing tool startups.
- Completeness vs. noise: capturing every keystroke or log line creates noise. Capture bounded artifacts (pre/post prompts, diffs, exit codes) and provide filtering controls for teams.
- Provider diversity: support for many CLI agents and browser assistants requires a plugin-like adapter model rather than hard-coding each tool.
Common failures and troubleshooting checklist
- No events appear after install — confirm hooks were installed with the two shell commands and the agent process is running.
- Diffs missing or incomplete — ensure the capture agent can access the repository and run git commands; check repository state (clean vs. unstaged changes) and permission issues.
- Proxy capture shows only headers — verify the proxy is configured as a local forwarding proxy for the process and that TLS interception (if used) is trusted by the client.
- Browser extension not recording — confirm extension permissions and that the assistant page is supported by the extension’s capture rules.
Practical use cases and outcomes
- Faster code reviews: share expiring, email-bound session links showing prompts and diffs so reviewers see full context for changes.
- Searchable tribal knowledge: hybrid semantic and keyword search over normalized session timelines recovers previous debugging sessions, regex solutions and refactors.
- Audit and compliance: stored prompts and diffs provide traceability for security reviews and incident investigations when properly redacted and access-controlled.
- Cost and usage analysis: a normalized timeline enables later attribution and analytics (covered in Part 2), but capture is the required first step.
Short FAQ
- Does this require changing app source code? No — the recommended pattern captures from the developer environment with native hooks, proxies and browser extensions.
- Will it expose secrets? Potentially, if unrestricted. Use redaction, sampling, and strict access controls; expire shared links.
- Does it work with any CLI agent? A plugin/adapter model is needed. The approach supports many agents but requires adapters for each runtime.
Next steps
Implement local hooks, proxy capture and a browser extension to create a unified, provider-neutral timeline. Validate installs with a short checklist (trigger edit, check diffs, verify browser and proxy events) and add redaction rules before enabling org-wide access. Part 2 will cover analyzing usage, cost attribution and detecting workflow friction once a capture corpus exists.
