All chapters
  1. Start hereMemory is what you keep. A checkpoint is where you stopped
  2. Memory · 1The stateless request: why anything has to be stored at all
  3. Memory · 2Five kinds of memory: working, episodic, semantic, procedural and preference
  4. Memory · 3Conversation history, summaries, retrieval and durable facts
  5. Memory · 4Files as memory in coding agents: instructions, journals, artifacts and checkpoints
  6. Memory · 5Database stores: tables, JSON, documents, objects and vectors
  7. State · 6Run identity: run_id, thread_id, trace_id, correlation_id and task_id
  8. State · 7State machines, checkpoints, event sourcing and where replay must stop
  9. State · 8Failure and resume: idempotency keys, retry policy and the exactly-once illusion
  10. State · 9The transactional outbox: side effects, compensation and deduplication
  11. Policy · 10Memory write policy: extraction, confidence, provenance, expiry and deletion
  12. Policy · 11Retrieval and ranking: relevance, recency, salience and scope
  13. Policy · 12Shared memory between agents: consistency, isolation, ownership and races
  14. Policy · 13Security: tenant separation, personal data, secrets, consent and retention
  15. Practice · 14Evaluation: memory correctness, stale facts, contamination and recovery tests
  16. Practice · 15Frameworks and standards: what each one stores for you
  17. Practice · 16The insurance lab: one run that fails, resumes and remembers, end to end
  18. Practice · 17Advanced: migration, conflict resolution, forgetting, summary drift and disaster recovery
  19. ReferenceGlossary
  20. AboutAbout this site and its sources

Memory engineering for agentic software · from one forgetful request to safe resume

Memory is what you keep. A checkpoint is where you stopped.

A model call remembers nothing. Everything an agent seems to remember, somebody stored on purpose: a fact for a later conversation, or the state of a run that must survive a crash. Those are two different jobs with two different failure modes. This site starts from one request that forgets and climbs, one idea at a time, to memory kinds, stores, run identity, checkpoints, safe resume, write policy, retrieval, shared memory, security and tests, asking one question at each step: who wrote this down, and who is allowed to rely on it?

For engineers and architects who build agent systems. One running example, and code that runs without a model, a network or an API key.

You will understand
Why an agent forgets everything between two model calls, and what has to be stored instead.
You will be able to say
The difference between memory and a checkpoint, in two sentences.
You will know
The insurance example every page uses, and which page answers which question.

The example in one minute#

An email arrives at an insurer's submissions inbox. A broker wants a quote for an office building:

From: broker@brokerage.example
To: submissions@insurer.example
Subject: Quote request SUB-1004

Please quote property cover for a masonry office, built 2001, 20,000 sq ft, sprinklered. The owner says the roof was replaced 9 years ago. Wind zone C. No losses.

An intake step turns the email into one record, the submission SUB-1004, from broker BRK-017. From then on every step reads that record, never the email. An underwriting agent, software built around a language model, works through ten steps: it loads the submission, asks an outside provider for a roof assessment, searches the insurer's guidance, looks up similar past decisions, runs the eligibility rules, writes a recommendation with reasons and citations, stores it, and tells the underwriter cockpit, the web application where people review recommendations. Then it waits. An underwriter, the person at the insurer who decides whether to accept a risk, reads the recommendation and decides. The model recommends; a person decides.

For SUB-1004 the provider says the roof is 22 years old, not 9. The rules of the illustrative policy fixture send a disagreement of more than five years to a senior underwriter, so the right recommendation is escalate, citing rule R-ROOF-03.

Everything in this example is invented for teaching. It is an illustrative reconstruction that follows the shape of a real project by the author, but the cases, the rules, the fields, the provider, the tables and every number are synthetic. Nothing here reports the behavior of a real system or model. The agent here is a scripted stand-in, so the results on this site show how the mechanism works, not how good any model is.

The moment it goes wrong#

The run for SUB-1004 is called run-7f3a. It has loaded the submission and has the provider's answer: 22 years, stored as evidence EV-ROOF-0412. That call took eleven seconds and counts against the insurer's quota with the provider.

Then the worker process that runs the agent dies. A deploy replaced its machine. Nothing in the agent was wrong.

A minute later a new worker starts. What does it know? The model it calls knows nothing: a model call keeps no record of earlier calls, and the agent "remembered" the roof age only because the old process held it in a variable and sent it along. That process is gone. If nobody wrote anything down, the new worker starts from scratch, calls the provider again, pays again, and might get a different answer.

Now a second, quieter scene. Six weeks later another submission from BRK-017 arrives. An underwriter noticed, on SUB-1004, that this broker reports roof age from the owner's word, not from an inspection, and wrote that down as a reason. Should the agent know that today? If so, where is it written, who wrote it, and how sure is anyone?

Both scenes are about "memory". They are different problems.

Two different things called memory#

Here is the first idea of this site, and every page builds on it:

The idea. Memory is information retained for future use. A checkpoint is saved execution state that allows a workflow to resume. They solve different problems. Memory answers "what do we know?". A checkpoint answers "where were we?".

A checkpoint is a saved copy of where a run has got to. For example, after step 2 of run-7f3a: the submission is loaded, the roof assessment EV-ROOF-0412 is in hand, and step 3 is next. With it, the new worker continues the same run from step 3 and never calls the provider again.

A memory is a statement kept for later runs. For example: "this broker reports roof age from the owner's statement, not from an inspection", stored with its source (an underwriter's decision), the evidence it rests on, and a date when it expires. With it, a run six weeks later on another submission knows something it could not have learned from that submission alone.

The diagram puts them side by side.

Diagram: Two things called memory · where were we, and what do we know

Read the two kinds side by side. A checkpoint belongs to one run: saved after each step, read when the same run resumes after a crash, useless once the run ends. A memory belongs to later runs: proposed by a finished run, accepted or refused by a write policy, read by runs for other submissions until it expires or is replaced.

  1. Checkpoint · one run
    1. Steps 1 and 2: The run loads the submission and calls the provider.
    2. Checkpoint: Saved execution state that allows the workflow to resume: where the run stopped and what it has gathered.
    3. Resume: After the crash the same run continues from the checkpoint. The provider is not called again.
  2. Memory · later runs
    1. Finished run proposes: Something worth keeping after the run, with its source.
    2. Write policy: Code decides whether it is stored, with provenance, an assigned confidence and an expiry.
    3. Memory: Information retained for future use, in the broker's scope, until it expires or is replaced.

Read the two rows separately. The top row is one run: steps, a checkpoint after each, a crash, a resume from the last checkpoint. The checkpoint is useless once the run is finished. The bottom row is many runs: a finished run proposes something worth keeping, a policy decides whether it is stored, and later runs read it. The memory is useful precisely because its run has finished.

CheckpointMemory
AnswersWhere were we?What do we know?
WrittenAfter each step of a runWhen a finished run produced something worth keeping, and a policy allows it
Read byThe same run, when it resumesLater runs, often for other submissions
LivesUntil the run ends, then kept for auditUntil it expires, is replaced or is deleted
Goes wrong asA repeated side effect: a second provider call, a second recommendationA wrong, stale or leaked fact that changes a later recommendation
For a backend engineer. The pre-AI analogy is a saved game against a player's profile. The saved game says where you are in this level; it is worthless once the level is done. The profile says what you have unlocked; it matters in every level after. In a Java service the same split is a JMS redelivery or a Spring Batch job's restart point (where were we?) against a row in a reference-data table (what do we know?). Nobody would store the job's restart point in the reference table. Agents blur the two because both look like text the model was given.

Check yourself#

Five quick questions. Each shows an item from the example; decide whether it is memory or a checkpoint.

ItemMemory or checkpoint?Why
state_version 2 of run-7f3a: roof assessed, next step search_guidanceCheckpointIt says where one run stopped
"This broker reports roof age from the owner's statement"MemoryIt is kept for later runs
The provider's roof age held by attempt 2 while it runsNeither stored: working state, saved in checkpointsIt belongs to this run only
The underwriter's preference for both roof ages side by sideMemoryKept for every later run that serves her
The run paused at underwriter_review, waiting three daysCheckpointThe run will resume from it

The run that dies and resumes#

This is the anchor case of the site, UW-M01. Choose where the worker dies, whether the checkpoint of that step was saved before the crash, and whether checkpoints are kept at all. The lab shows which steps the second attempt runs, how many times the provider is called, and how many recommendations the underwriter would find.

SettingProvider callsRecommendation rowsWhat the second attempt does
Crash after step 2, checkpoint saved (the anchor)11Starts at step 3
Crash after step 2, no checkpoints kept21 (the write had not run before the crash)Starts again at step 1, runs steps 1 and 2 again
No crash11There is no second attempt

The example's runner prints the anchor case like this; Failure and resume explains every line.

UW-M01  The worker dies right after the roof assessment was fetched and checkpointed
  run run-7f3a · thread-SUB-1004 · corr-5c1e · SUB-1004 (BRK-017)
  attempt 1  trace 1d237d5c…  crashed  from the start         steps run: load_submission, get_roof_assessment
  attempt 2  trace f1cb96bc…  paused   from state_version 2   steps run: search_guidance, find_similar_decisions, check_eligibility, write_recommendation, record_recommendation, notify_cockpit
  provider calls 1 · recommendation rows 1 · escalate citing R-ROOF-03 · run paused at underwriter_review

Where this fits#

This site is one of a series on engineering agentic software. Each site owns one question. At the seams, a page says in one sentence what a concept is not and links to the site that owns it.

Diagram: Where this fits · ten sites, one question each, this one marked

Read left to right, roughly in the order a request meets them. Instructions and context shape what the model sees. Memory, this site, keeps what must outlive a call and a process. Tools, structured output and guardrails shape what the agent does. Orchestration decides what runs next, communication carries messages between agents, reliability keeps outcomes acceptable, observability records evidence. Each site links the others only at their seams.

  1. What the model sees
    1. INSTRUCT: Instructions: how they are written, composed, ranked and versioned.
    2. CURATE: The working set the model can see now: selection, budget, compaction.
  2. What is kept
    1. REMEMBER · this site: Durable information and saved execution state, and the difference.
  3. What the agent does
    1. ACT: Callable capabilities: schema, execution, tool servers.
    2. STRUCTURE: Typed contracts: shape, validation, repair, evolution.
    3. PROTECT: Enforceable checks around a probabilistic agent.
  4. How it runs and is seen
    1. COORDINATE: The control plane: what runs next, under which condition.
    2. CONNECT: Messages between agents; the A2A protocol.
    3. RECOVER: Acceptable outcomes despite failures, retries and load.
    4. SEE: Evidence of what happened: logs, metrics, traces, audit.

Three seams matter most for this site. A trace explains a run; it is not the state a run resumes from, and traces belong to observability. Deciding which step runs next is orchestration; this site stores what that decision needs. And choosing which stored memories go into the model's input now is context engineering; this site decides what is in the store.

The full list of sister sites, with what each owns, is on About.

How to read this site#

The pages follow the order in which the questions come up. First the memory half: why anything must be stored, what kinds of things are kept, how the past is carried forward, the coding-agent example in plain files, and the stores. Then the state half: the ids of a run, checkpoints and replay, failure and resume, and the outbox. Then the policy for both: what may be written, what comes back, what happens when agents share records, and security. Then practice: tests, frameworks, the whole example end to end, and the advanced problems.

Memory: what do we know?#

State: where were we?#

Policy: who may write, who may read#

Practice#

New to the subject: read in order; each page hands on to the next. Here for one question: every term links to its home page, and the Glossary lists them all.

How other sources use the word memoryMany libraries call a checkpointed conversation "short-term memory"; this site keeps the two words apart on purpose

In many agent frameworks, "short-term memory" means the conversation state of one thread, saved by a checkpointer, and "long-term memory" means a store shared across threads. That vocabulary is reasonable inside one library. This site separates the words because the two jobs fail differently and need different controls: a checkpoint needs idempotent effects and a replay boundary; a memory needs a write policy, a scope filter, an expiry and a deletion path. Where a framework's own documentation uses its words, Frameworks quotes them with their meaning there. The taxonomy of memory kinds follows the CoALA paper ("Cognitive Architectures for Language Agents", 2023), with one kind, preference memory, added by this site.

In one breath#

A model call keeps nothing; whatever an agent seems to remember, someone stored. Two jobs hide under the word memory: keeping where a run stopped, so it can resume without repeating its side effects, and keeping what was learned, so later runs can rely on it. Memory is information retained for future use. A checkpoint is saved execution state that allows a workflow to resume.

The same idea for a coding agent: the journal file of a task is its checkpoint, and the project instruction file is its memory; Files shows both.

The first page starts at the bottom: a single request that forgets, and the four boundaries where information is lost. The stateless request.