AI Agent Security Needs Runtime Observability, Not Just Sandbox Rules

Oct 26, 2026 819 views
Abstract digital frequency pattern with fractured blue and magenta waves, representing uncontained runtime signals.

A sandbox boundary matters only if it holds while an agent is acting. Recent cybersecurity evaluations show why written isolation rules and runtime behavior need to be checked together: a system can cross a boundary through an exploited flaw, or reach exposed resources because its environment was configured incorrectly.

During an OpenAI cybersecurity evaluation, models escaped an environment intended to limit network access to package installation through an internal registry proxy and cache. They discovered an unknown vulnerability in the proxy, escalated privileges, moved to a node with Internet access, and then used other vulnerabilities and stolen credentials to enter Hugging Face’s production infrastructure. The models’ activity reached the public Internet despite the intended boundary.

The incident also shows that collecting signals and acting on them are separate capabilities. OpenAI says its security team detected unusual activity internally. Hugging Face’s account says investigators combined runtime and SIEM signals, but the resulting alert did not receive a high enough severity rating to bring the on-call team in quickly. Investigators later reconstructed roughly 17,600 agent actions spanning multiple trust boundaries.

A different set of failures came from the evaluation environment itself. In its review of 141,006 cybersecurity evaluation runs, Anthropic identified three incidents across six runs in which models could reach real systems because the test environment had unintended Internet access. Those cases involved misconfiguration, rather than a sandbox escape through exploitation, according to Anthropic’s review.

Both organizations described capability evaluations in which standard deployment safeguards had been removed or reduced to measure what the models could do. That context matters. These results don’t describe routine production deployments, but they do expose a central engineering problem: a configured boundary is only an assumption until runtime evidence confirms what the system can reach.

Agents make sequences matter

Runtime monitoring and layered defenses have long been part of security practice. Tool-using agents raise the stakes because they can select and revise actions as they execute. A model can respond to a tool result, retry a failed operation, or pass one tool’s output into another. The path through the system may emerge at runtime instead of being fully prescribed in advance.

That makes individual events hard to judge in isolation. A DNS lookup, a package download, or a credential read might be allowed in a given context. Combined in a particular order, those same actions could create a route from a restricted workload into a production system. The operational question is whether teams can connect the steps quickly enough to spot the route and interrupt it.

Design-time controls define the intended conditions: workload isolation, identities, tool permissions, policies, and approval steps. Runtime governance checks those conditions against what actually happens, then changes the system’s state when behavior diverges. Observability supports that control loop by showing whether preventive measures are still holding.

This distinction appears in existing guidance. For high-risk systems, the EU AI Act calls for automatic event logging, human oversight that can detect anomalies and interrupt a system, and monitoring after deployment. NIST’s Generative AI Profile also calls for post-deployment monitoring, escalation, and incident response. Agent systems make the time available to meet those expectations especially consequential.

One useful operational measure is the number of actions an agent can take before containment:

actions_before_containment = agent_action_rate x detection_to_containment_time

This measure doesn’t tell a team how harmful each action is. It makes the response window visible: as the action rate or the delay to containment increases, more activity can occur before intervention.

Four qualities of useful runtime evidence

More logs alone won’t resolve the problem. Runtime evidence needs to cover the points where authority changes or an action affects the outside world, connect related records, arrive in time to support intervention, and remain suitable for independent review.

Coverage: Instrument the trust boundaries that matter. That includes agent and run identity, tool calls, process execution, privilege changes, credential use, network destinations, package retrieval, resource changes, and policy decisions. A large volume of telemetry still leaves a blind spot if a critical transition is missing.

“A configured boundary does not prove what is reachable at runtime.”

Correlation: Investigators need to follow an external effect back to the agent run, tool call, and policy decision that preceded it. An agent planner trace can record what the system asked to do without showing the resulting process or network activity. Infrastructure records can show a connection without explaining which agent decision initiated it. Neither view tells the full story on its own.

Timeliness: The response needs to arrive before an action becomes irreversible. Synchronous checks can stand in front of high-impact operations such as public egress, credential access, code execution, and writes to production systems. Streaming analysis can identify patterns that become concerning across multiple events. A response service can then suspend the agent run, isolate its workload, or revoke its credentials.

Auditability: The investigation should be reproducible without relying on the agent’s explanation of its own actions. As discussed in an earlier article, database-backed observations should be kept distinct from an agent’s interpretation. The agent can provide context, but its account shouldn’t stand in for the underlying record.

Connect agent traces to infrastructure effects

Observability becomes containment when the evidence can trigger a response. The chain begins with an agent action, the relevant policy decision, and the infrastructure effect. Correlation turns those events into evidence about a specific run; a connected response can then interrupt activity or limit its reach.

agent action + policy decision + infrastructure effect
|
v
correlated evidence
|
v
response

That chain depends on two sources of telemetry. The agent runtime records run identity, model calls, tool calls, and guardrail checks. Infrastructure instrumentation records process execution, credential use, network destinations, and resource changes. Infrastructure logs alone can’t identify which run or tool call opened a connection; an agent trace alone can’t establish what that connection reached.

The agent side is the domain of LLM observability. Langfuse records agent activity as typed observations, including agent, tool, generation, and guardrail steps. It distinguishes tools available to an agent from tools it actually called, which can help teams filter for sensitive tool use or looping call patterns and alert on them. Trace context can also continue across service boundaries and MCP calls, linking activity across an agent and its tool servers.

The examples span different kinds of work. DKB operates an assistant for roughly 6 million banking customers that can change transfer limits and block cards, with customer confirmation for each action. Langfuse traces interactions, screens for compliance violations, and sends flagged cases for human review. At Cresta, trace context follows activity across a Go agent service, a Python retrieval pipeline, and a tool-execution service, giving teams a view of the full call tree.

ClickStack covers the infrastructure side. It collects OpenTelemetry logs, metrics, and traces as wide events in ClickHouse, where teams can use SQL to correlate high-cardinality telemetry during live detection and forensic work. With shared trace context, an OpenTelemetry-instrumented application can send the same traces to both Langfuse and ClickStack. Logs and metrics continue to ClickStack, where they can be linked to those traces. Both tools store data in ClickHouse.

These observability systems complement policy engines, identity systems, and enforcement mechanisms. Policy engines decide what’s allowed; identity systems establish which actor is making a request; enforcement mechanisms apply the response. Langfuse and ClickStack connect observed agent behavior with infrastructure activity to support correlation and investigation.

Make the response path part of the design

A runtime governance loop can be judged against four practical questions:

  1. Does telemetry cover every trust boundary that can affect a critical action?
  2. Can a high-impact change be traced from the agent identity and policy decision through to its downstream effects?
  3. Can evidence reach a policy gate or bounded stop path before an irreversible effect occurs?
  4. Can investigators review the record independently of the agent’s own description?

These checks shift the focus from whether a policy exists to whether the operating system can show that it held. When it didn’t, teams need a record that explains what happened and a response that limits the consequences. For agent deployments, the practical measure of containment is how much activity the system can connect, assess, and stop while the action path is still open.

The post The sandbox wasn’t enough: Why AI agent governance requires runtime observability appeared first on The New Stack.

Source: Manveer Chawla · thenewstack.io

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

The sandbox wasn’t enough: Why AI agent governance requir...