A single request to fix a failing test can trigger multiple model calls, file reads, searches, command executions, test retries, and branches before an AI agent ever proposes a patch, producing hundreds or thousands of individual data points in the process. An analysis published by The New Stack examines how execution records from AI coding agents are forcing development teams to rethink their data storage strategies. The report concludes that for many agent-based products, what starts as diagnostic telemetry quickly becomes application data that users expect to retrieve, review, and retain indefinitely.
The volume challenge stems from how agent work differs from traditional software tasks. Pull request volume scales with patches submitted, and issue volume scales with development tasks, but agent traces scale with the execution graph inside each task. Each step in an agent's workflow can produce its own span or event, and adding a new tool, retry policy, or branch increases data volume even when the number of completed tasks stays flat. In one case study cited, Laminar documented more than 500,000 browser events per day, with a single browser-agent session running for over 30 minutes and generating hundreds of thousands of DOM diff events. The firm used those events to reconstruct a video-like replay of what the agent observed during execution.
The report identifies three distinct signals that indicate traces have outgrown the primary database. First comes contention, where ingestion or retention work consumes enough I/O and CPU to affect transactional operations. Next is analytical friction, where evaluations and debugging queries need to scan long time ranges or join large trace tables and stop meeting latency targets. Eventually, teams resort to forced sampling, discarding traces to protect the application database even though the product or an audit process requires the complete record. Langfuse, an open-source platform for LLM observability, documented both problems as it scaled, experiencing Postgres IOPS exhaustion during ingestion and prompt API latency reaching seven seconds under heavy load.
The analysis explains that the storage problem exists because agent traces carry dual requirements: one person loads one trace to understand a single run, while the engineering team scans many traces to find patterns across thousands of executions. This combination of point retrieval and cohort analysis gives the data its unusual shape. Most records are written once rather than updated, since a model invocation or tool result describes an event that already happened, but they carry high-cardinality dimensions such as model version, prompt template, tool name, session ID, user ID, and outcome. Langfuse moved its tracing data from Postgres to ClickHouse while keeping transactional and latency-sensitive paths isolated, then collapsed separate trace, observation, and score tables into a wide, mostly immutable observations table. Initial table loads for large datasets went from seconds to milliseconds, and dashboard load times for large projects improved by at least 10 times over longer time ranges. The report recommends starting with the reads your product must support, then choosing the simplest architecture that meets those requirements, keeping traces in the primary database at modest volume and sending trace events to a dedicated analytical store as contention grows. For teams building agent products, the boundary between diagnostic telemetry and durable application data shifts the moment users need to retrieve, display, or retain an execution record. The architectural choice between operational simplicity and analytical performance will only sharpen as agents handle more complex, longer-running tasks across enterprise workflows.