Agent Observability Has a Compression Problem

Long-running agents produce more evidence than humans can review. Enterprise observability must compress their trajectories without breaking auditability.

An agent investigates a service incident. It reads tickets, retrieves documents, queries monitoring systems, tries a fix, checks the result and hands the case to a person.

The run leaves behind prompts, tool calls, retrieved passages, retries, intermediate outputs and changes to external systems. Multiply that by hundreds of cases and the organisation has a new problem: plenty of recorded activity, very little human capacity to inspect it.

The trace is available. The explanation still takes work.

Recording more is only half the answer

Agent observability is becoming a product category in its own right. LangChain connects tracing with evaluation and debugging; OpenTelemetry is developing conventions for agent and tool operations.[1][2] These are useful foundations for seeing what a system actually did.

But collecting a trajectory and reviewing it are different jobs.

A developer investigating one failed run can open individual steps. A process owner reviewing a day's activity needs to see outcomes, exceptions and decisions that require attention. An auditor may later need to reconstruct one particular action and the authority under which it occurred.

Those readers need different levels of detail over the same evidence.

The market opportunity, in my view, is the layer between the recorded run and the person responsible for it: a reviewable account that stays connected to its underlying records.

A dashboard is already an act of compression

Every operational view leaves something out.

A success rate removes individual cases. A case summary removes intermediate steps. An alert groups many events into one signal. Even a trace viewer compresses activity through labels, collapsed branches and filters.

The question is therefore not whether to compress agent activity. Human oversight depends on doing so.

The question is which distinctions must survive compression.

Consider a procurement agent that selects a supplier. A summary says:

Supplier B selected within the approved budget.

That may be accurate. It may also conceal the only detail that matters: Supplier A was cheaper, but its security review was incomplete; Supplier B required an exception approved for this purchase only.

Without those distinctions, a reviewer cannot tell whether the agent followed the process, bypassed it or established a precedent it had no authority to establish.

A short account can be factually correct and operationally misleading.

Compress the review, preserve the evidence

A useful design separates three things:

The review view is a derived product. It should be possible to regenerate it from retained records and inspect the evidence behind each material claim.

That does not require keeping every byte forever. It requires an explicit retention policy, stable references and a clear indication when supporting content has been redacted, expired or was never captured.

Nor does it require exposing a model's private chain of thought. The accountable record concerns observable actions, supplied evidence, declared justifications, permissions and results. An agent's explanation is itself a claim to verify.

A summary should help someone reach the evidence. It should not quietly become its replacement.

What needs to survive

For a consequential business action, I would expect the compressed account to retain at least:

• Outcome and effect: what the agent changed, where, and whether the result was checked.
• Authority: which instruction, policy or approval permitted the action, including its scope.
• Evidence: which source versions and tool results supported the decision, with resolvable references.
• Exceptions: failed checks, conflicting information, retries with changed assumptions and unresolved uncertainty.
• Human involvement: what was presented for approval, who approved it, and what happened afterwards.

The difference between “approval requested” and “approval received” must survive. So must the difference between a successful API response and a verified business outcome.

These details may occupy only a few lines. Losing one can change the meaning of the entire run.

Two compression problems, two tests

There is a related engineering problem inside the agent itself.

Long-running agents may compact their working context so they can continue within a bounded context window. Anthropic describes this technique and warns that aggressive compaction can lose subtle information whose importance emerges later.[3]

That is working-memory compression. Its immediate test is whether the agent can continue the task correctly.

Compressing a trajectory for human oversight is review compression. Its test is whether a person can assess the run and reconstruct a material action.

A compacted conversation may help the agent continue without being an adequate audit record. A detailed audit archive may preserve the run without being a useful briefing for a manager.

Enterprise systems need to design and evaluate both.

Test what the summary hides

Summary quality cannot be measured by readability alone.

Take a set of recorded runs and ask reviewers to assess them first through the compressed view, then through the underlying evidence. Include ordinary successes as well as failures and exceptions.

Can they identify an action taken without approval? Can they distinguish a temporary exception from a standing permission? Can they find a failed validation that occurred after a successful tool call?

Then measure:

• Material omissions: which decision-relevant facts disappeared.
• Unsupported claims: which summary statements lack supporting evidence.
• Reconstruction time: how long it takes to locate the records behind a disputed action.
• Review burden: how much human time the view saves at an acceptable level of detection.

These are proposed acceptance criteria, not established industry benchmarks. Their thresholds depend on the process and the consequences of missing an exception.

Sampling remains necessary. If the same system performs the work and decides what reviewers should see, its omissions can hide its own failures. Independent checks should compare selected summaries with their source records.

The Market Signal

The first purchase question is often: can we trace the agent?

The next enterprise question should be: can our people review its work at the volume we intend to run?

A product that records everything but demands manual inspection of every step moves the bottleneck to the reviewer. A product that offers a polished narrative without an evidence trail makes review easier while weakening accountability.

The useful middle ground is layered observability: a concise operational view, explicit exceptions and a dependable route back to the relevant records.

For buyers, that means testing a vendor's summary against a difficult real case, including a disputed approval or a failed validation. For builders, it means treating evidence references, summary versions and omission checks as product capabilities.

As agents work for longer and across more systems, human oversight will depend increasingly on what their observability layer chooses to leave out.

The competitive question is how much review effort it can remove while preserving the evidence needed to challenge a decision.

Sources

1. LangChain — Context Engineering: context management, tracing and evaluation.
2. OpenTelemetry — Semantic conventions for GenAI agent and framework spans: developing conventions for agent and tool operations.
3. Anthropic — Effective context engineering for AI agents, September 29, 2025: compaction and its information-loss tradeoffs.

The layered review model and acceptance criteria above are the author's analysis and recommendations.