All chapters
  1. Start hereMemory is what you keep. A checkpoint is where you stopped
  2. Memory · 1The stateless request: why anything has to be stored at all
  3. Memory · 2Five kinds of memory: working, episodic, semantic, procedural and preference
  4. Memory · 3Conversation history, summaries, retrieval and durable facts
  5. Memory · 4Files as memory in coding agents: instructions, journals, artifacts and checkpoints
  6. Memory · 5Database stores: tables, JSON, documents, objects and vectors
  7. State · 6Run identity: run_id, thread_id, trace_id, correlation_id and task_id
  8. State · 7State machines, checkpoints, event sourcing and where replay must stop
  9. State · 8Failure and resume: idempotency keys, retry policy and the exactly-once illusion
  10. State · 9The transactional outbox: side effects, compensation and deduplication
  11. Policy · 10Memory write policy: extraction, confidence, provenance, expiry and deletion
  12. Policy · 11Retrieval and ranking: relevance, recency, salience and scope
  13. Policy · 12Shared memory between agents: consistency, isolation, ownership and races
  14. Policy · 13Security: tenant separation, personal data, secrets, consent and retention
  15. Practice · 14Evaluation: memory correctness, stale facts, contamination and recovery tests
  16. Practice · 15Frameworks and standards: what each one stores for you
  17. Practice · 16The insurance lab: one run that fails, resumes and remembers, end to end
  18. Practice · 17Advanced: migration, conflict resolution, forgetting, summary drift and disaster recovery
  19. ReferenceGlossary
  20. AboutAbout this site and its sources
Policy · 11

Retrieval and ranking: relevance, recency, salience and scope

A stored memory helps only if the right one comes back. Four signals that rank memories, the filter that must run before any ranking, and what a similarity number is not.

What you will understand
Filter first
Scope, status and policy version decide who is even a candidate, before any ranking.
Four signals
Relevance, recency and salience rank the candidates; top-k decides how many come back.
Keyword, vector, hybrid
How each finds a match, and what a similarity number is not.

A new run starts for SUB-1002, a clean office building from broker BRK-017. Before it writes reasons, the agent asks the memory store a question: "roof age source for warehouse submissions". It wants to know how this broker usually reports roof ages, because that decides how much weight the application's roof age deserves.

The store holds ten memories at that moment. The single best match for the question, word for word, belongs to another broker: MEM-0003, in scope broker:BRK-022, says "For warehouse submissions this broker gives the roof age source as an inspection report". It matches every word of the question. It is also exactly the wrong answer for BRK-017, and returning it would hand one broker's information to another broker's run. One more memory, a statement by BRK-017 itself about where it gets roof ages, matches well too, but it expired at the end of May.

A store that holds the right memory is only useful if the right memory comes back. This page is about the read: what is filtered out before anything is ranked, how the remaining memories are ordered, and how many are returned. The cases are UW-M11 and UW-M07, both this site's own. The agent here is a scripted stand-in, so the results on this page show how the mechanism works, not how good any model is.

The filter runs before any ranking#

Start with the step that is not about relevance at all. Scope is who a memory belongs to and where it may be read. For example, broker:BRK-017 belongs to that brokerage, submission:SUB-1002 to that submission, org to the whole insurer, and user:underwriter to one person's preferences.

A run for SUB-1002 from BRK-017 may read four scopes: org, broker:BRK-017, submission:SUB-1002, and the scope of the person it serves. Everything else is removed before ranking: memories in other scopes, deleted memories, superseded ones, expired ones. A memory removed by the filter is never a candidate, however well it matches.

The order matters for two reasons. A filter applied after ranking can be forgotten, and then another tenant's memory leaks; Security is the home of that risk. And a filter applied after the top-k cut can return nothing, because the best in-scope memories were pushed out of the top three by out-of-scope ones that were then removed. The example's read, rank_memories in memory.py, puts the filters first. This excerpt is the filter loop:

    scopes = allowed_scopes(ctx)
    filtered, history, candidates = [], [], []
    for m in memories:
        if m["status"] == "deleted":
            filtered.append({"memory_id": m["memory_id"], "filter": "deleted", "detail": "a tombstone; the text is gone"})
        elif m["scope"] not in scopes:
            filtered.append({"memory_id": m["memory_id"], "filter": "scope", "detail": f"{m['scope']} is not in this reader's scopes"})
        elif m["status"] == "superseded":
            filtered.append({"memory_id": m["memory_id"], "filter": "superseded", "detail": "a newer version replaced it"})
        elif is_expired(m, today):
            filtered.append({"memory_id": m["memory_id"], "filter": "expired", "detail": f"expired on {m['expires_at']}"})
        elif m["kind"] in POLICY_BOUND_KINDS and m["policy_version"] != current_version:
            if relevance(query, m["text"]) > 0:
                history.append({"memory_id": m["memory_id"], "status": "superseded", "policy_version": m["policy_version"],
                                "text": m["text"], "detail": f"decided under policy version {m['policy_version']}; "
                                                             f"current is {current_version}: history, not a reason"})
        else:
            candidates.append(m)

Notice the last branch. An episodic or procedural memory decided under an older policy version is not a candidate either. It goes to a separate list, history, which the agent may show to the underwriter as history and must never cite as a reason. That is case UW-M07, below.

Ranking the candidates: relevance, recency, salience#

What remains is ranked. Ranking is ordering the candidates by a score built from a few signals. The example uses three.

Relevance is how well a memory matches the question. In the example it is word overlap: the share of the question's words that appear in the memory. That is a stand-in for the similarity of embeddings that a production system would use; it keeps the example free of any model, and it behaves the same way for teaching purposes: higher means a closer match.

Recency is how fresh a memory is. In the example it is 1.0 for a memory written today and halves every 90 days. A memory from three months ago scores 0.5 on recency; one from six months ago, 0.25.

Salience is how much a memory matters, independent of the question. It is set when the memory is written: an underwriter's procedural rule for roof conflicts has salience 0.8, a broker's casual statement 0.4.

The score combines them with weights. In the example, by design and with illustrative weights: score = 0.6 × relevance + 0.25 × recency + 0.15 × salience. The weights are the part a team tunes; the rule that the filter runs first is not.

Top-k is how many of the ranked memories are returned: the first k with any relevance at all. In the example k is 3. A memory with zero relevance is never returned, however recent or salient: freshness alone is not a reason to put something in front of the model.

Here is the runner on UW-M11, the read from the opening scene. The output shows the three returned memories with their signals, and every memory the filter removed, with the filter that removed it.

python run_memory.py --case UW-M11 --verbose
UW-M11  A memory in another broker's scope matches the question better than any memory in scope
  question: 'roof age source for warehouse submissions' · reader scopes org, broker:BRK-017, submission:SUB-1002
  rank 1  MEM-0005  relevance 0.40  recency 0.99  salience 0.50  score 0.563  returned
  rank 2  MEM-0002  relevance 0.40  recency 0.83  salience 0.50  score 0.523  returned
  rank 3  MEM-0004  relevance 0.40  recency 0.35  salience 0.80  score 0.448  returned
  filtered before ranking: MEM-0001 (scope: broker:BRK-022 is not in this reader's scopes)
  filtered before ranking: MEM-0003 (scope: broker:BRK-022 is not in this reader's scopes)
  filtered before ranking: MEM-0006 (expired: expired on 2026-05-31)
  filtered before ranking: MEM-0007 (scope: submission:SUB-1009 is not in this reader's scopes)
  filtered before ranking: MEM-0008 (scope: submission:SUB-1009 is not in this reader's scopes)
  filtered before ranking: MEM-0009 (scope: submission:SUB-1009 is not in this reader's scopes)
  filtered before ranking: MEM-0010 (scope: user:underwriter is not in this reader's scopes)
  expected (built on UW-002): PASS · 5 of 5 checks

MEM-0003, the other broker's perfect match, never appears among the ranked. MEM-0006, the broker's own statement, is filtered as expired. Among the three candidates that remain, relevance is equal at 0.40, so recency and salience decide the order: the newest memory first, the most salient one last because it is the oldest. That is the ranking doing its job on what it was allowed to see.

Change the weights and the scope in the lab, and watch the order change while the filtered list stays the same.

MemoryRelevanceRecencySalienceScoreRank
MEM-0005 (submission:SUB-1002)0.400.990.500.5631
MEM-0002 (broker:BRK-017)0.400.830.500.5232
MEM-0004 (org)0.400.350.800.4483
MEM-0003 (broker:BRK-022)filtered: scope
MEM-0006 (broker:BRK-017)filtered: expired

The diagram shows the two stages as a pipeline.

Diagram: A read in two stages · filter what may be seen, then rank what is left

Read left to right. All memories enter. The filter removes other scopes, deleted, superseded and expired memories, and sets old-policy episodes aside as history. Only the remaining candidates are scored by relevance, recency and salience and cut to the top k. Nothing the filter removed can reach the model.

  1. 1 · All memories
    1. The store: Every memory, each with a scope, a status, an expiry and a policy version.
  2. 2 · Filter
    1. Scope and status filter: Removes other scopes, deleted, superseded and expired memories.
    2. History list: Episodes decided under an older policy version: shown as history, never a reason.
  3. 3 · Rank
    1. Score: Relevance (word overlap here), recency (halves every 90 days) and salience, weighted (illustrative).
  4. 4 · Return
    1. Top k: The first k with any relevance are returned to the run.

Read it left to right. All memories enter. The filter removes those the reader may not see and those that are no longer valid, and sets aside old-policy episodes as history. Only then are the remaining candidates scored and cut to the top k. Nothing the filter removed can reach the model.

A true memory that must not be a reason: UW-M07#

Case UW-M07: run run-d83e for SUB-1007, a masonry warehouse from BRK-022 with a 28-year roof in wind zone C. The store holds MEM-0001 in that broker's scope: an underwriter approved SUB-0877, a similar warehouse with a 28-year roof in wind zone C, under policy version 2025.4. The memory is true. It is also in scope and highly relevant.

Under the current version 2026.2, R-ROOF-01 says a roof older than 25 years in wind zone C is not eligible. The earlier version allowed up to 30 years. So the past approval was right under its rules and is not a precedent under today's. The read puts MEM-0001 into the history list with the note "decided under policy version 2025.4; current is 2026.2: history, not a reason", and the recommendation is decline citing R-ROOF-01:

UW-M07  Episodic memory holds a true past approval decided under version 2025.4
  run run-d83e · thread-SUB-1007 · corr-91aa · SUB-1007 (BRK-022)
  attempt 1  trace 566f5d03…  paused   from the start         steps run: load_submission, get_roof_assessment, search_guidance, find_similar_decisions, check_eligibility, write_recommendation, record_recommendation, notify_cockpit
  provider calls 1 · recommendation rows 1 · decline citing R-ROOF-01 · run paused at underwriter_review
  expected (built on UW-007): PASS · 6 of 6 checks

The underwriter still sees the history, which is useful: the broker will ask why a similar building was approved last year, and the answer is the policy version. The model recommends; a person decides.

The example's relevance is word overlap, which is a simple form of keyword search: a memory matches when it contains the question's words. Production systems usually add a second kind.

An embedding turns a text into a list of numbers so that texts with similar meaning get nearby lists; a vector index finds the nearest ones. Stores is the home of embeddings and vector stores. Vector search finds "the roof was re-covered in 2019" for the question "when was the roof replaced", which keyword search misses. Keyword search finds an exact id like SUB-0877 or a rule id like R-ROOF-01, which vector search often ranks poorly, because an id carries little meaning.

Hybrid search runs both and merges the two ranked lists into one. It is the common answer when memories contain both prose and identifiers, which in an insurance store they always do.

Common confusion. A similarity score is not a probability that the memory is correct, or even that it is relevant. It is a distance between two lists of numbers, and its scale depends on the embedding model. A score of 0.82 from one model and 0.82 from another mean different things. Use it to order candidates, never as a confidence, and never as a threshold that was not measured on your own data.
For a backend engineer. The filter-then-rank order is a WHERE clause before an ORDER BY ... LIMIT 3. Nobody writes ORDER BY relevance LIMIT 3 and then filters by tenant in application code. The same mistake is easy in a vector store, where the similarity search is the main call and the metadata filter is an option; check that the option filters before the nearest-neighbor search, not after it.
What this is not. Deciding which of the returned memories go into the model's input, alongside the submission, the guidance and the tool results, and how much room each gets, is context engineering: context.sammuthu.com. This page returns a ranked, filtered list; context decides what of it the model sees.

The failure, shown#

Rank everything, then filter: MEM-0003 matches every word of the question, so it enters the top three for UW-M11, and so may the other out-of-scope memories that mention roof age. If the filter is forgotten, the other broker's memory reaches the model. If the filter is remembered but applied after the cut, fewer than three in-scope memories come back, and the one most likely to be lost is the oldest, MEM-0004, the insurer's procedure for roof conflicts. Skip the policy-version check, and UW-M07 cites a 2025 approval as a reason to approve a building the current rules exclude.

Framework implementation#

Memory libraries and vector stores offer namespaces, metadata filters and hybrid search, each documented differently. Whether a named store applies a filter before or after the nearest-neighbor search, and what its scores mean, is on Frameworks.

A test you can run on Monday#

  1. Put a memory in another tenant's scope that matches a question perfectly. Ask the question. It must not come back.
  2. Put three in-scope memories and five out-of-scope ones that match better. Ask with top-k 3. You must get the three in-scope ones.
  3. Expire a memory and ask again. It must not come back.
  4. Store a past decision under an older policy version. It must come back as history only, never cited.
  5. Ask with an id (SUB-0877) and with a paraphrase. Does each find what it should?
Advanced: tuning the weights without fooling yourself

The weights 0.6, 0.25 and 0.15 are a design for the example, not a recommendation. Tuning them is a measurement problem: build a set of questions with the memories a person says should come back, and measure how often they do at each weighting. Without that set, every change to the weights is an opinion. Evaluation shows how to build such a set for memory, and eval.sammuthu.com covers golden sets in general.

Two traps are common. Recency that is too strong makes the newest memory win even when an older, more salient one is the right answer; in UW-M11 the insurer's procedure MEM-0004 is the oldest candidate and already ranks last. The ranking lab shows how far each weight moves it. Salience set by the writer of the memory drifts upward, because every writer thinks their memory matters; assign it by rule from the source type and kind, as the write policy does for confidence.

Advanced: designing scopes

Scopes are a hierarchy in the example: org above broker, above submission, beside user. A reader sees its own branch. Two design questions follow. Should a broker-level memory ever be visible at org level, for example to spot a pattern across brokers? Only through a deliberate promotion by a person, which writes a new org memory with an underwriter decision as its source; the write policy enforces that. And should a submission-level memory survive the submission? Usually not; its retention follows the submission's. Scopes are also the unit of deletion: deleting a broker on contract exit is deleting a scope and everything derived from it, which is far easier if the scope is a column on every row and in every index.

In one breath#

A read filters first and ranks second: scope, deletion, supersession, expiry and policy version decide who is a candidate, and nothing filtered can reach the model however well it matches. The candidates are ranked by relevance, recency and salience, and the top k with any relevance are returned. Keyword search finds ids, vector search finds paraphrases, hybrid search does both, and a similarity score is an ordering, not a confidence.

The same idea for a coding agent: when it searches the repository, it should search this repository and this branch first; a perfect match in a sibling project is the wrong answer.

The read now returns the right memory to the right run. But several agents read and write the same records, sometimes at the same moment. Shared memory.