A memory library’s job is not to store everything. It is to store the right things and retrieve the relevant things.

Memori is a .NET library that adds persistent memory to any IChatClient — recalling relevant facts before each model call and capturing the full conversation afterward. What sets it apart is the architectural split between conversation history and searchable facts, each with its own storage contract.

That distinction drives the entire abstraction design.

If you store too much, recall becomes noisy, storage costs grow, and the signal-to-noise ratio degrades. If you store too little, recall returns empty and the memory layer becomes useless.

The design problem is not technical. It is semantic. What counts as “memory” depends on how it will be used later. Conversation history is durable archive. Extracted facts are searchable knowledge. Summaries are compressed context. Each has different access patterns, different consistency requirements, and different storage characteristics.

Memori splits persistent storage into two concerns because of this — and the split is the most important architectural decision in the library.

The two-storage split

The first version of Memori had a single IStorage interface. It covered conversations, entities, embeddings, and raw retrieval in one monolithic contract. Every storage provider had to implement everything, even if it only cared about one part.

The feedback was consistent: teams using a relational database for conversations and a vector database for facts had no clean way to express that separation. A single IStorage forced either an awkward adapter or a lowest-common-denominator implementation that did neither side well.

The split resolved that.

IConversationStorage — the append-heavy side

IConversationStorage handles ordered, relational data:

  • entities and processes (the actors)
  • sessions (grouping boundaries)
  • conversations and their messages (the durable history)
  • conversation summaries (rolling compression)

The operations are append-mostly: create entities, start sessions, append messages, retrieve history in insertion order. There is no vector search here, no embedding math, no relevance scoring. It is a straightforward append-and-retrieve contract.

var entityId = await storage.GetOrCreateEntityAsync("user_123");
var sessionId = await storage.GetOrCreateSessionAsync("session_abc", entityId, processId);
var conversationId = await storage.GetOrCreateConversationAsync(sessionId, timeout);

await storage.AppendMessagesAsync(conversationId, messages);
var history = await storage.GetConversationMessagesAsync(conversationId);

The contract is intentionally narrow. Implementation expectations are:

  • Safe for concurrent use
  • GetOrCreate methods are idempotent for the same logical identifier
  • Messages are stored in insertion order without reordering or deduplication
  • Each operation is internally atomic — the interface does not expose transactions

The built-in InMemoryConversationStorage serves as the reference implementation and default for development. Production backends implement the same interface against PostgreSQL, Cosmos DB, SQL Server, or whatever your team already uses.

VectorStoreCollection<string, MemoryFactRecord> — the search-heavy side

Facts — the searchable memory — live in a standard Microsoft.Extensions.VectorData collection. Any VectorStore provider works directly. No Memori-specific adapter needed. This is the same vector abstraction used by Semantic Kernel — analogous to Spring AI’s VectorStore for Java readers.

var store = new InMemoryVectorStore();
var facts = store.GetCollection<string, MemoryFactRecord>("memori_facts");

await facts.UpsertAsync(new MemoryFactRecord
{
    Id = Guid.NewGuid().ToString("N"),
    EntityId = entityId,
    Content = "User prefers concise explanations",
    Embedding = embedding,
    MemoryType = "preference",
    CreatedAt = DateTimeOffset.UtcNow
});

var results = await facts.SearchAsync(queryEmbedding, new VectorSearchOptions
{
    Top = 10,
    Filter = new FilterClause().EqualTo(nameof(MemoryFactRecord.EntityId), entityId)
});

MemoryFactRecord carries content, embedding, confidence, memory type (fact, semantic triple, process attribute, summary), scope for isolation, and version metadata. It is the domain object for everything recall touches.

The split between IConversationStorage and VectorStoreCollection is not an implementation detail. It is the library’s way of saying that conversation history and searchable memory are different problems with different solutions. History is append-ordered and needs durability. Facts are vector-indexed and need fast approximate search. Mismatching those concerns is how memory layers become slow, expensive, or both.

Attribution, sessions, and capture

Memory needs to belong to someone.

Attribution identifies the entity (user, agent, system) that facts and conversations belong to. It is the primary partition key for recall and management operations.

memori.Attribution("user_123", "support_agent");
memori.SetSession("session_abc");
  • Attribution sets the owning entity and optional process. Recall and delete operations are always scoped to the current entity.
  • Sessions group capture history. GetOrCreateConversationAsync uses SessionTimeout to decide whether to continue an existing conversation or start a new one.
  • Capture persists conversation messages. It runs after the model responds, so the full input-output pair is stored together.

Capture policy gives the application control over what gets persisted:

options.ExcludedCaptureRoles.Add(ConversationRoles.Tool);
options.DropEmptyMessagesOnCapture = true;
options.CaptureMessageFilter = message => message.Role != "developer";
options.CaptureMessageTransform = message => new ConversationMessage(
    message.Role,
    message.Content.Replace("secret-token", "[redacted]", StringComparison.OrdinalIgnoreCase),
    message.Type,
    message.CreatedAt,
    message.Metadata);

Roles can be excluded, empty messages dropped, messages filtered by predicate, or transformed before storage. The defaults are conservative — only system messages are stripped — but production deployments typically tighten these rules.

One lesson here: the library originally had no capture policy. Every message went to storage. The first production deployment showed that tool messages, function call inputs, and internal annotations quickly dominated storage volume without adding recall value. The capture policy was added not as a feature but as a necessity.

The recall pipeline

Recall is where the library earns its keep. The pipeline has a straightforward shape:

Query text
    ↓
EmbeddingGenerator (optional)
    ↓
VectorSearch + Lexical fallback
    ↓
Ranking (similarity × confidence × recency)
    ↓
Relevance threshold filter
    ↓
Prompt context formatting

Step 1: Embedding

If an IEmbeddingGenerator<string, Embedding<float>> is registered, the query text is embedded and used for vector search. If not, recall falls back to lexical search against the VectorStoreCollection. This means the library works without embeddings — useful for demos, tests, or scenarios where exact text matching is sufficient.

Vectors and embeddings sound like ML-researcher territory, but they have become routine infrastructure. .NET ships System.Numerics.Tensors.TensorPrimitives for vector math — SumOfSquares, Multiply, Divide — right in the base class library. Both DeterministicEmbeddingGenerator and NgramEmbeddingGenerator in this library use these primitives: hash tokens into a float[], then L2-normalize with TensorPrimitives. No deep learning background required. Both generators can be written in minutes by describing what you need to a coding agent connected to Microsoft Learn MCP — the agent searches the docs, finds the right APIs, and produces the implementation. This is the new reality: seasoned software engineers can reach into ML-adjacent territory and be productive immediately.

Built-in generators include DeterministicEmbeddingGenerator (fixed vectors, zero dependencies) and NgramEmbeddingGenerator (character n-gram based, reasonable for short text). Production deployments bring their own via IEmbeddingGenerator.

VectorStoreCollection.SearchAsync handles the backend query. The implementation varies by provider, but the contract is the same: return the top-N results for a vector or text query, optionally filtered by entity ID.

Step 3: Ranking

IMemoryRanker is the extension point for combining multiple signals into a final score. The default implementation combines:

  • Base similarity or rank score from the search
  • Confidence boost (0–1, configurable per fact)
  • Recency boost decaying logarithmically with age
public interface IMemoryRanker
{
    double Rank(RecallResult result, DateTimeOffset now);
}

Custom rankers can boost specific memory types, penalize old facts, or apply domain-specific logic. This is one of the small extension points that repays the investment quickly in production.

Step 4: Threshold

Results below RecallRelevanceThreshold are dropped. The default is 0.15 — chosen because the original default of 0 bypassed relevance filtering entirely. That made recall appear to work during prototyping while silently returning junk in production. The change was one line in MemoriOptions, but it prevented a class of bugs that every new user hit.

A RelaxedMode flag overrides the threshold to 0 for debugging and development.

Step 5: Formatting

Results are rendered as a <memori_context> block for prompt injection:

<memori_context>
Only use the relevant context if it is relevant to the user's query.
Relevant context about the user:
- The user's favorite color is blue. Stated at 2026-05-06 01:00:00
</memori_context>

Formatting is configurable — bullet style, timestamp format, heading text, whether to include summaries — through MemoriOptions.

Middleware semantics

MemoriChatClient wraps any IChatClient and adds recall-before and capture-after behavior. The contract is straightforward but specific:

  1. Recall runs before the model call. Fact context is injected as a system, developer, or user message depending on configuration.
  2. Injecting context does not persist it. The injected <memori_context> message is not stored as conversation history.
  3. Capture runs after the model call completes. Both standard and streaming responses are captured, including streaming reconstruction.
  4. Provider metadata is preserved. Response metadata from the AI provider is copied to assistant messages with memori.provider.* keys.

Injection placement is configurable:

  • BeforeHistory — first message in the request (default)
  • AfterSystemAndDeveloperMessages — after existing instructions
  • AppendToRequest — last message before the user’s input
  • MergeIntoFirstSystemMessage — merge into an existing instruction message
  • Disabled — capture only, no injection

The Disabled option deserves special mention. Some teams want capture without injection — they maintain their own prompt construction but want Memori to handle storage and recall. That option exists because we learned that not every team wants the library to touch their prompt pipeline. The middleware should make the common case easy and the uncommon case possible.

Request-scoped control via MemoriRequestOptions allows per-call overrides of recall and capture without changing the shared MemoriOptions. This matters for applications that serve multiple contexts within the same facade instance.

Augmentation as background extraction

Augmentation runs after capture, extracting structured memory from conversation messages. It is designed as a cold path — it runs asynchronously, not inline in the chat pipeline.

public interface IAugmentationClient
{
    ValueTask<AugmentationResult?> AugmentAsync(
        AugmentationInput context,
        CancellationToken cancellationToken = default);
}

The input includes entity ID, conversation ID, captured messages, and the previous summary. The output can contain four kinds of structured memory:

  • Facts: standalone, entity-scoped statements designed for future recall. When a user says “I want everything in dark mode,” augmentation produces {content: "User prefers dark mode", memoryType: "preference", confidence: 0.92}. These are upserted into the VectorStoreCollection and become directly searchable by the recall pipeline.
  • Semantic triples: subject-predicate-object relationships — "User" → "prefers" → "dark mode" — that enable graph-style queries across sessions, surfacing connections that flat text search would miss.
  • Process attributes: lightweight string tags for the current workflow — "onboarding-flow-complete" or "tier-1-support" — used for routing, state tracking, or conditional behavior.
  • Conversation summary: a compressed replacement for the full conversation history. When provided, it replaces the previous summary in IConversationStorage, keeping long-running conversations within context-window limits.

The built-in PromptAugmentationClient sends the conversation to an IChatClient with a structured extraction prompt and parses the JSON response. It ignores malformed output gracefully — better to miss a fact than to crash the pipeline.

The NullAugmentationClient is the default: a no-op for hosts that only want capture and recall. Augmentation is opt-in by design.

One of the trickier lessons here was idempotency. Augmentation runs after every capture. If a host retries a request, replays history, or uses background workers, the same logical memory may be generated more than once. The library does not enforce deduplication — that responsibility falls to the augmentation client and storage provider. Deterministic extraction and stable keys are the recommended approach. In practice, consecutive turns produce overlapping rather than identical extractions, so the real risk is not exact duplicates but near-duplicates with slightly different scores. The library favors correctness here: let the storage layer resolve conflicts through upsert semantics, and treat a few redundant facts as preferable to missing one entirely.

A useful rule of thumb

Conversation storage is your history. Fact storage is your memory. Do not conflate them.

History is append-ordered, durable, and rarely searched semantically. Memory is vector-indexed, scored, and queried by relevance. If you are storing facts in the same system that stores raw conversation messages, you are paying for capabilities you do not need on one side and missing capabilities you do need on the other.

What comes next

These abstractions — storage split, recall pipeline, middleware semantics — are enough to build a working memory layer. But production deployments need answers to harder questions:

  • How do you isolate facts between tenants without leaking memory across workspaces?
  • What happens when two concurrent writes arrive for the same fact — and which version wins?
  • How do you keep long-running conversations coherent when raw history grows past the model’s context window?
  • What APIs do you expose to end users who want to see, edit, or delete their stored memories?
  • Can you span multiple storage backends — hot facts and archival — without changing application code?

These are production patterns: optional, backward-compatible, and the subject of the final post.

For a closer look at the code and examples, check out the Memori repo: https://github.com/khurram-uworx/Memori

NuGet: https://www.nuget.org/packages/Memori

Getting started: https://github.com/khurram-uworx/Memori/blob/main/GETTING-STARTED.md

Architecture and design notes: https://github.com/khurram-uworx/Memori/blob/main/ARCHITECTURE.md

More from Memori