Retrieval¶
Purpose¶
Store embedded documents and answer top-k queries — the search half of retrieval-augmented generation — through a two-method port, with an offline in-memory implementation and a ready-made agent tool.
When to use¶
When an agent should answer from a corpus that is not in the prompt:
product docs, policies, tickets, code. You already have an
embedding.Embedder; this package stores what it produces and ranks
queries against it. When retrieval should stay out of the model's
hands — you always know which passages a task needs — call the index
directly instead of registering the tool. Large-scale or persistent
corpora belong in a real vector database: it implements the same
retrieval.Index interface and the rest of the application is
unchanged.
How it works¶
Chunk your corpus (your job — section boundaries and overlap are
corpus-specific), embed the chunks with EmbedDocuments, and add them
with their vectors:
index := retrieval.NewMemory()
embeddings, err := embedder.EmbedDocuments(ctx, texts)
// one retrieval.Embedded per chunk: Document{ID, Text, Metadata} + Embedding
err = index.Add(ctx, embedded)
hits, err := index.Search(ctx, queryVector, 5) // best first
Search ranks by cosine similarity, ties broken by document ID, so
equal scores order deterministically. A zero vector scores 0 rather
than NaN. Dimension mismatches fail with *retrieval.DimensionError —
the usual symptom of indexing and querying with different embedding
models. Add replaces by ID, so re-indexing a changed document is
idempotent. Implementations snapshot: mutating your slices afterwards
does not touch the index, and hits carry copied metadata.
As an agent tool, the same index becomes one option:
agent, _ := golem.New[struct{}, string](client, decoder,
golem.WithTools[struct{}, string](
retrieval.MustNew[struct{}](retrieval.ToolConfig{
Index: index,
Embedder: embedder, // the same model that embedded the corpus
K: 5,
}),
),
)
The search_documents tool takes {"query": string}, embeds the
query with EmbedQuery, and returns the hits as ranked text — ID,
score, text, metadata — best first. Malformed arguments are correctable
*model.ModelRetry; an embedding provider failure is a plain error,
because a quota problem is not a model mistake.
Example¶
Run examples/retrieval — offline: a deterministic keyword stand-in
plays the embedder, a four-section policy corpus is indexed, and the
model searches it as a tool to answer a support question. Swap the
stand-in for a real embedder to go live.
API surface¶
retrieval.Index—Add(ctx, docs []Embedded) error,Search(ctx, query []float32, k int) ([]Hit, error)retrieval.NewMemory() *Memory— the offline in-memory cosine index;Len()reports its sizeretrieval.Document{ID, Text, Metadata},retrieval.Embedded{Document, Embedding},retrieval.Hit{Document, Score}retrieval.DimensionError{Expected, Got, ID}— the typed width mismatchretrieval.New[Deps](retrieval.ToolConfig{Index, Embedder, K}) (tool.Tool[Deps], error),retrieval.MustNew[Deps](...)— thesearch_documentstoolretrieval.ToolName,retrieval.DefaultK—search_documents, 5
Gotchas¶
- Index and tool must share one embedding model. Cosine scores are only comparable within a model; the dimension error catches width mismatches, but two models with the same width fail silently and confidently.
- Chunks, not whole documents. The embedder's contract says chunking is yours; a 50-page PDF embedded as one vector retrieves as one blob.
Memoryis linear-scan and in-process: right for tens of thousands of chunks, wrong for millions or for multi-service access. ImplementIndexagainst your vector database; the tool does not change.- Metadata is returned verbatim with every hit and rendered into the tool's output — do not put secrets there.
- Re-adding an ID replaces the document. That is the update path; it is not incremental — search sees the whole replacement.
- Scores are similarities in [-1, 1], not percentages; 0 means unrelated, not absent. Threshold in your own pipeline if you filter.
- The design — two-method port, the zero rule, the tool's retry boundary — is ADR 0040.