An AI agent's logs do not prove that it did anything

If an AI agent has gained access to the production environment, its own log cannot be considered proof. A compromised or simply erratic agent may distort the account of its actions, and a monitor running in the same container would share that same vulnerability.

The main conclusion drawn by the authors of NOFire is inconvenient for anyone accustomed to viewing trace data as the ultimate source of truth: an audit depends not only on what is recorded, but also on the vantage point from which it is observed.

The farther away the observer, the higher the trust

The authors present a three-tiered hierarchy. The first tier consists of the agent’s transcript, its tool calls, and its own description of the session. This is the most readable layer, but the agent can read and modify everything that is accessible to it.

The second level is the observer inside the guest system. They have a better view of processes, files, and connections, but they are still within the attack’s reach: a fully compromised virtual machine could theoretically deceive them as well.

The third layer is the virtual machine’s boundary. Packets, disk operations, and outbound traffic from the VM are logged from the host side, which the guest cannot access. This log is less informative, but it cannot be overwritten from within.

This doesn’t mean you should discard standard logs. Their role simply changes: the agent’s account becomes a hypothesis, observations inside the VM serve as additional evidence, and events at the boundary act as a verification against which both layers must be cross-checked.

The Most Useful Signal Is a Contradiction

If the agent reports that it has only read the configuration, but the external log shows an outgoing connection, that is already sufficient reason not to trust the session. It doesn’t matter whether the agent lied, made a mistake, or misrepresented the action: the discrepancy is recorded by a signal that the agent could not have faked.

This is precisely where an audit shifts from reading a polished narrative to conducting an investigation. A match between the layers increases confidence, while a discrepancy helps quickly pinpoint where verification is needed.

But there’s a practical catch. Raw records like “connection to an address” or “a series of I/O operations” answer the question of facts, but not the question of “what the agent did.” NOFire proposes solving this through “intent lifting”: mapping characteristic patterns of network activity and I/O to operations such as cloning a repository, installing a dependency, or making an API call, while specifying confidence separately.

The idea makes sense precisely because agents operate through a repetitive set of actions. But this is not a magical reconstruction of intentions: classification remains an interpretation of the facts, not the facts themselves.

Where Confidence Ends

The authors claim that boundary observation is passive, does not interfere with the hot path, and is inexpensive enough to enable it for every task in production. However, comprehensive measurements and methodology are promised in a separate publication, and no quantitative results are presented here.

Other open questions remain: how accurately does the taxonomy reconstruct intentions, how does the system behave when layers conflict in real incidents, and where is the line between useful aggregation and overly bold interpretation?

The practical advice from this analysis is simple: don’t choose between agent logs and infrastructure telemetry. Keep both layers, clearly separate facts from narratives, and escalate any discrepancy to the incident level. For a postmortem, this is much more honest than the phrase “the agent reported that it did everything correctly.”

If you had to investigate an AI agent failure today, would you trust its logs, or would you start gathering evidence outside its sandbox in advance? Source: Hacker News - Newest: “AI” “LLM”