The underwriting agent is halfway through submission SUB-1004. It has read the submission, which says the roof is 9 years old. It has called the outside provider, which says the roof is 22 years old. That call took eleven seconds and counts against the insurer's quota with the provider. The roof assessment is stored as evidence EV-ROOF-0412, and the agent is about to search the insurer's guidance.
Then the worker process that runs the agent is restarted. A routine deploy replaced the machine image. Nothing went wrong in the agent; the platform did its job.
When the new process starts, what does it know about SUB-1004? If nobody wrote anything down, it knows nothing. It does not know that the provider was called, so it will call again and pay again. It does not know the provider's answer. The model it calls knows nothing either: a model call keeps no record of earlier calls. This page is about that blank: why it exists, where exactly information is lost, and the four questions to ask before you decide to store something. The case is UW-M01, the anchor of this site's own cases.
The agent here is a scripted stand-in, so the results on this page show how the mechanism works, not how good any model is.
A model call keeps nothing#
Start with the smallest piece. A model call is one request to a language model and one response. Everything the model may use must be in the request. When the response has been returned, the call is over and nothing about it stays with the model.
That property has a name. A stateless service keeps nothing between requests: each request carries everything needed to answer it. For example, an HTTP endpoint that converts a temperature from Celsius to Fahrenheit is stateless; it needs no memory of your previous conversion.
Model APIs work this way. Anthropic's documentation for its Messages API, read for this page, says it directly: "The Messages API is stateless, which means that you always send the full conversational history to the API." A chat that seems to remember your earlier question remembers it because the application sent the earlier question again, inside the new request.
The text the model can see in one call is its context window: everything in the request plus the response being written. The same documentation calls the context window a "working memory" for the model and distinguishes it from the data the model was trained on. Everything in the request counts toward it: the instructions, every message, every tool result and the definitions of the tools.
What this is not. Choosing what goes into the context window, how much room each part gets, and what to drop when it is full is context engineering, and it has its own site: context.sammuthu.com. This site starts one step earlier: what must be kept somewhere so that it can be put into a later request at all.
So the first lesson is short. A model does not remember. An application remembers on its behalf, by storing information and sending it again.
Where information is lost: four boundaries#
Information in an agent system lives in several places at once, and each place has its own lifetime. The useful way to see this is to name the boundary that ends each one.
The first boundary is the end of a call. The model's view of the request disappears. In UW-M01, the model that read the roof assessment while writing a reason no longer has it.
The second boundary is the end of a conversation, or of a session. A session is one continuous interaction that an application keeps together, such as one chat window or one run of a command-line tool. For example, a broker chatting with an assistant about SUB-1003 across twelve messages is one session. When the session ends, whatever the application held only for that session is gone unless it was stored.
The third boundary is the end of a process. Values that live only in the variables of a running program are in-process state. For example, a Python dictionary holding the roof assessment while the worker runs.
A crash is the process ending without warning: a restart, an out-of-memory kill, a deploy, a power cut. In-process state does not survive it.
The fourth boundary is the loss of a machine or a disk. Anything written to a local file on that machine is gone with it. Only data written to a store that keeps copies elsewhere survives.
Information that survives the end of the process and a restart of the machine is called durable.
Persistence is the act of making it durable: writing it to a store that outlives the process. In this site "durable" and "persistent" mean the same thing; durable is the word used from here on.
The diagram puts the four boundaries in a row, from the shortest-lived holder to the longest.
Diagram: Four boundaries · each one wipes out the holders to its left
Read left to right, from the shortest-lived holder to the longest. The model input ends with the call. The session ends with the conversation. Process variables end with the process. A local file ends with the disk. Only a durable store with copies elsewhere survives all four boundaries.
- Ends with the call
- Model input: Everything sent in one request. Gone when the response returns.
- Ends with the conversation
- Session: What the application keeps for one continuous interaction, unless it stores it.
- Ends with the process
- Process variables: Values in the running worker's memory. A crash or a restart ends them.
- Ends with the disk
- Local file: Survives a process restart; lost with the machine or its disk.
- Survives all four
- Durable store: A database or object store that keeps copies on other machines.
Read it from left to right. Each holder survives every boundary to its left and is wiped out by the boundary under it. The model input ends with the call. The application's session ends with the conversation. Process variables end with the process. A file on the local disk ends with the disk. Only a durable store with copies elsewhere survives all four.
Try it in the lab: pick a boundary and see which holders still have the roof assessment of SUB-1004 afterwards.
| Holder | Call ends | Conversation ends | Worker restarts | Machine or disk lost |
|---|---|---|---|---|
| Model input | Lost | Lost | Lost | Lost |
| Process variables | Survives | Survives, if the process lives on | Lost | Lost |
| Checkpoint store (a database) | Survives | Survives | Survives | Survives, if the database has copies elsewhere |
| Memory store (a database) | Survives | Survives | Survives | Survives, if the database has copies elsewhere |
| Object storage with backup | Survives | Survives | Survives | Survives |
The table answers the opening question. When the worker restarted in UW-M01, only the database survived. If the roof assessment had been held only in process variables, it was gone, and the restarted worker would call the provider again.
For a backend engineer. None of this is new. A Spring service's request-scoped beans end with the request, the HTTP session ends with the user's session, the heap ends with the JVM, and only rows committed to the database survive a redeploy. An agent adds one more holder at the far left, the model input, which ends after every single call. What is new is how much an agent does between two commits: minutes of tool calls and model calls, each one slow, some of them paid for, all of them lost if the process dies in the middle.
Two different reasons to store something#
Look again at what the restarted worker needed in UW-M01. It needed two different things, for two different reasons.
It needed to know where the run was: that steps 1 and 2 were done, and that step 3 was next. Without that, it starts the run over and calls the provider a second time. That is saved execution state. A checkpoint is a saved copy of it, written after a step, so that the run can continue from there.
It might also, next month, need to know what was learned: that broker BRK-017 reports roof age from the owner's statement, so a provider check on its submissions is worth the cost. That is information retained for future runs, which this site calls memory.
The two are easy to confuse because both are written to a database, both survive a crash, and both are called "memory" in casual speech. They answer different questions. Memory is information retained for future use. A checkpoint is saved execution state that allows a workflow to resume. A checkpoint is useless once its run has finished; a memory is useful precisely because its run has finished.
| Checkpoint | Memory | |
|---|---|---|
| Question it answers | Where were we? | What do we know? |
| Written | After each step of a run | When a run has produced something worth keeping |
| Read | When the same run resumes | By later runs, often for other submissions |
| Lives | Until the run ends, then kept for audit | Until it expires, is replaced or deleted |
In UW-M01 | state_version 2: roof assessed, step 3 next | None yet; written at the end of the run |
Four questions before storing anything#
Storing is not free. Every stored item has to be protected, kept consistent, eventually deleted, and trusted by someone who did not write it. So before an item becomes durable, answer four questions.
- What exactly is stored? The roof assessment record, with its evidence id, or a sentence a model wrote about it? These are different items with different risks.
- For whom? This run only, every run for this submission, every run for this broker, or the whole insurer? The answer is the item's scope.
- For how long? Until the run ends, for 30 days, until the next policy version, or for seven years for audit?
- Who may rely on it? May a later run use it as a reason in a recommendation, or only show it as history? May it change an outcome?
For the roof assessment of UW-M01 the answers are: the provider's record with id EV-ROOF-0412; for this run, and as evidence for this submission; until the submission is closed and then for the audit period; and the agent may cite it as evidence, while only an underwriter decides. Those answers say it belongs in the evidence store and in the run's checkpoint, and not in the long-term memory of the broker. The rest of the site turns each question into machinery: kinds of memory, stores, ids, checkpoints, write policy, scope and retention.
The failure, shown#
Here is the same restart with and without a checkpoint. The numbers are from the example's scripted run and are illustrative.
| No durable state | Checkpoint after each step | |
|---|---|---|
| After the restart, the worker knows | Nothing about SUB-1004 | Steps 1 and 2 are done; the roof assessment is EV-ROOF-0412 |
| Provider calls for this submission | 2 | 1 |
| Steps run twice | 1 and 2 | None |
| What the underwriter eventually sees | One recommendation, after a second paid call | One recommendation |
The first column is not wrong because of a bug. Every component did what it was built to do. It is wrong because nobody decided what had to survive a restart. Failure and resume shows the full mechanism; for now it is enough to see that the cost lands on the second outside call and on time.
Framework implementation#
Agent frameworks differ mainly in what they make durable for you without asking. Some save a checkpoint after every step automatically; some keep only the conversation; some keep nothing unless you configure a store. What each named framework stores is read from its own documentation on Frameworks. Until you have read that, assume nothing outlives the process.
A checklist for Monday#
Take one agent you run and walk through these.
- Kill the worker process in the middle of a run. What does the restarted process know? Write it down before you try.
- Which steps would run again? Which of those call something outside, cost money, or write something?
- For every item the agent holds, name the boundary that ends it: call, conversation, process or disk.
- For every item you store, answer the four questions. Which items have no answer to "who may rely on it"?
Advanced: why stateless calls are a feature, not a limitation
A stateless model API is easy to scale and to reason about. Any request can go to any server, a failed request can be sent again, and nothing in the provider's systems has to be cleaned up when a user leaves. The price is paid by the application: it must own every piece of state and send what is needed again on each call.
For agents this price shows up in three places. Cost and latency grow with what is sent again on each call, which is the subject of context engineering. Correctness depends on the application's state being right, which is the subject of this site. And security depends on what the application chooses to keep, which is covered on Security.
Some providers and frameworks offer to keep conversation state on the server for you. Treat any such offer as a convenience with its own lifetime and its own deletion rules, read in its documentation, not as a durable store for your run state or your memory. A run that must survive a crash should not depend on a cache it does not control.
Advanced: how durable is durable?
"Survives a restart" is a spectrum. A file on a local disk survives a process restart and dies with the disk. A single database server survives both until its disk fails. A database with synchronous replication to another zone survives the loss of a machine. A backup in object storage survives the loss of the database, with a recovery point in the past. Each step up costs money and latency on every write.
For an agent the right level differs by item. Run state needs to survive a worker restart, which almost any database gives. Evidence and decisions need to survive the loss of a database, because the insurer must be able to show later what an underwriter saw. Memory sits between: losing it degrades later runs, but no run fails without it. Choosing the level per item, instead of one level for everything, is the first real design decision this site asks for. Backup and restore are covered on Advanced.
In one breath#
A model call keeps nothing; an application remembers by storing information and sending it again. Information is lost at four boundaries: the end of a call, of a conversation, of a process and of a disk, and only a durable store survives all four. There are two reasons to store something: to continue a run after a crash, which needs a checkpoint, and to know something in a later run, which needs memory. Before storing anything, answer what, for whom, how long, and who may rely on it.
The same idea for a coding agent: when a session ends, the model forgets the whole session; only what was written to a file or committed to git is there next time.
Once something is worth keeping, the next question is what kind of thing it is: what the run is doing now, what happened before, what is true, how work is done, or what a person asked for. Five kinds of memory.