Durable Records Let AI Agents Resume Payment Operations Safely

Oct 28, 2026 929 views

3D abstract illustration of illuminated purple and blue ribbon waves folding across a dark, textured field.

A crashed agent can leave a payment action in the worst possible state: dispatched, but not confirmed. The request may have reached the provider and completed, or it may have failed before arriving. If a replacement agent treats that uncertainty as a fresh task, it could issue a second refund. The balance may allow it, so the duplicate might not trigger an obvious error.

That makes agent recovery a question of operational control, not simply restarting a process. The next agent needs to inherit the specific approved operation, know what remains uncertain, and be constrained from inventing a substitute action. Its reasoning can help investigate; durable application state must define what it’s allowed to do.

“The replacement must inherit the original operation, not reinterpret it.”

Make the operation durable before dispatch

The agent’s process, variables and transient reasoning context can disappear. The approved refund cannot depend on any of them surviving. Before the application sends a request, it should write an authoritative operation record to a transactional store.

That record needs to identify the approved amount, authorization, operation ID, status and idempotency key. The application should also attach its operation ID to the request as provider metadata, where supported. Together, these fields give a replacement a stable reference to the original work instead of asking it to reconstruct intent from a conversation or trace.

Record the operation as created, then mark it dispatched when the request leaves. Move it to confirmed or failed only after receiving a provider response. If execution stops while the record says dispatched, the replacement knows a request was attempted but the application has no recorded outcome. That status is a signal to investigate, not permission to retry.

At the tool boundary, the application should validate every proposed action against the committed record. The agent may suggest what to do next, but the dispatch layer decides whether the request matches the approved operation, including its parameters and identity. That separation matters because an agent can reason differently after a restart; the application’s enforcement should not.

In the architecture described here, ClickHouse Managed Postgres stores the authoritative business record. Native change data capture (CDC) replicates that data into ClickHouse. Application telemetry goes to ClickStack, while agent traces go to Langfuse; both use the same ClickHouse service as their backend database.

Putting operational events and traces alongside replicated business data can make an incident easier to investigate. With a shared operation ID, an engineer can connect the refund record, the agent run and a failed HTTP call in one query. The identifier must be propagated deliberately into the database row, trace attributes and service log context; the shared backend doesn’t create that relationship by itself.

There’s also a hard boundary around authority. CDC can lag by seconds, so the replacement must read the current operation from Postgres before taking action. The replicated copy helps with analysis, but the application should enforce permissions using the authoritative row it committed.

System architecture diagram showing an Original Agent and Replacement Agent interacting through an Application Tool Boundary with ClickHouse Managed Postgres and a Payment Provider, with execution logs, traces, and CDC data syncing to ClickHouse for investigation.

Resuming a workflow doesn’t settle an external effect

Execution frameworks can preserve progress and restart work after a failure. Durable execution tools such as LangGraph and Temporal recommend making side effects idempotent. That advice addresses repeated calls, but restarting a step doesn’t reveal whether a provider completed the first request before the process failed.

The durable operation record gives an agent a fixed identity to carry through recovery. Reusing the recorded idempotency key lets a provider recognize a retry as the same logical request, when the provider supports that behavior. It also covers a race in which the original request completes after the agent has crashed or a worker assumed dead resumes later. AWS describes the broader pattern in its Builders’ Library: the caller supplies an identifier that is recorded atomically with the mutation.

Yet a key cannot answer every question. A mutation can produce an external effect even when the caller receives an error; Stripe documents that a 500 response may still accompany a completed effect. When the result remains uncertain, the replacement needs evidence and a recovery protocol, not a new guess.

Langfuse can show whether the earlier agent invoked the refund tool and which idempotency key it used. ClickStack can show service events around the same period, such as an outbound request followed by a connection failure. The Langfuse MCP server and ClickStack MCP server can expose those records to the replacement, queried by operation ID.

Keep those investigative tools read-only. They can help narrow the time and likely failure point, while the application’s tool boundary retains control over payment actions. Neither logs nor traces prove that the provider committed a refund. An absent trace is inconclusive too: Langfuse sends traces asynchronously, and a crash may prevent delivery. The payment provider remains the source of truth for its own transaction.

Use a recovery protocol, then recheck authority

Once the replacement has the original operation record and has gathered available evidence, its next move should follow an application-defined recovery protocol. If the provider confirms completion, record the result and continue. If the provider reports that the refund is pending, wait for a final outcome; sending another request could duplicate the effect.

If the provider confirms that the request failed without an external effect, the application can consider a new authorized attempt after checking the relevant preconditions. If the provider’s records don’t resolve the question, escalate. A missing refund entry doesn’t establish that the request was never processed; the transaction may still be unsettled.

“A stored approval is historical evidence, not current authority to act.”

Even a confirmed outcome doesn’t settle whether another action remains appropriate. The refundable balance may have changed, another representative may have issued a separate refund, the account may have changed status, or the authorization may have been revoked. Before a new mutation, check current conditions and authorization rather than treating the old approval as permanently valid.

The practical test for replaceable agents is whether a new process can recover the objective, outstanding obligations and uncertain side effects from durable records, then proceed within application-enforced limits. Agent memory can aid the investigation, but it shouldn’t carry the authority to act. Persist the work, correlate its evidence and make the provider confirm the result; then a replacement can continue without turning a crash into a duplicate transaction.

The post AI agents need to be replaceable – without losing your work appeared first on The New Stack.

Source: Manveer Chawla · thenewstack.io

Comments

Sign in to comment.
No comments yet. Be the first to comment.

Related Articles

AI agents need to be replaceable – without losing your work