Tuesday, April 7, 2026

Deterministic replay is non-negotiable for agents

Hooksbase
Reliability
Replay lineage in the Hooksbase dashboard

Replay sounds simple: take a failed delivery, send it again. For classical SaaS webhooks it almost is — the consumer is deterministic, same payload in, same response out, retry until it works.

For agents, replay is a different problem. The consumer is non-deterministic. LLM output varies. Tool calls have side effects. The same payload can produce different results on every run. That means the event layer under an agent has to make exactly one thing deterministic: the input bytes the agent sees.

If the input drifts, everything downstream drifts. And most event stacks let it drift in at least three ways.

Drift #1: the payload changes between attempts

A consumer that does its own serialization can mutate the payload between retries. Hooksbase persists the raw source payload the moment it arrives, before any routing or transform runs. Every retry of a delivery reads from that stored payload. Same bytes, every attempt.

Drift #2: the transform config changes between the original run and the replay

This is the subtle one. Say your agent receives Stripe events through a Handlebars transform that extracts customer_id, amount, and line_items. Three weeks later you tweak the transform to also extract currency. Nothing wrong with that — the agent is getting better.

Now a customer asks you to replay a failed event from last week, while its payload is still inside the retention window. Without snapshotting, the replay would apply the current transform to the old source payload. The agent would receive a shape it never got the first time. Maybe it handles the new field. Maybe it doesn't. Either way, the replay isn't a replay — it's a new run with a new input.

Hooksbase persists a dispatch snapshot alongside every transformed delivery. The bytes that were dispatched to the destination on the original run get saved separately. Replay reads the snapshot, not the current transform. While the payload is retained, the destination sees the same bytes it saw the first time.

Drift #3: the routing decision changes

Routing rules match on the source payload. If you change the rules after an event is accepted, the delivery still gets routed per the original evaluation — the resolved destination config is snapshotted on the delivery record.

Retries, replays, and DLQ re-drives all read from that snapshot. So an event that originally went to the "billing-automation" destination still replays to billing automation, even if you've since reordered the routing table.

Why this matters concretely

Think about the incidents you actually have with agent workflows:

  • Agent timed out on 50 events overnight. Want to replay them without fear of sending different inputs.
  • LLM provider had an outage, 200 events dead-lettered. Want to use Starter+ bulk replay once service is back.
  • Customer reports their event didn't trigger the agent. Want to manually replay that single delivery and confirm the behavior.
  • You changed the agent prompt yesterday. Want to re-test against a captured replay of real production events from last week.

All four need the exact same input bytes. Without dispatch snapshots, three of them quietly produce different results than the original and you don't notice until someone reports a weird discrepancy.

What Hooksbase persists per delivery

Every delivery keeps:

  • the raw source payload
  • the transformed dispatch snapshot (if a transform ran)
  • the resolved destination config on the delivery record
  • every attempt with request and response details
  • replay lineage via replay_of_delivery_id
  • DLQ row for terminal failures
  • file references for email or form attachments

The one caveat

Replay doesn't make the agent itself deterministic. If the agent calls an LLM, it's still going to get different text on each run. That's fine — the agent's non-determinism is contained. The event layer's job is to make sure the input to that non-determinism is reproducible.

Which is exactly what you want. Replay becomes "run the same input again with this new agent version" instead of "run some unknown approximation of the original input with the current agent and hope for the best."

Keep reading