/data

The corpus, the models and the results

The corpus

6 articles of the SQuAD v1.1 development set (Rajpurkar, Zhang, Lopyrev and Liang, EMNLP 2016), licensed CC BY-SA 4.0: Computational complexity theory, Packet switching, Steam engine, Oxygen, Prime number, Apollo program. 37,499 Qwen2.5 tokens in 1,234 sentences. Of their 1,160 questions, 200 were picked with a seeded generator (seed 23); each keeps its first gold answer and that answer's character offsets. Changes: paragraph newlines replaced by spaces; paragraphs joined by a blank line; first gold answer kept. Source file SHA-256 95aa6a52d5d6a735563366753ca50492a658031da74f301ac5238b03966972c9. The derived corpus is distributed under the same licence.

The models (run once, offline)

Embeddings: sentence-transformers/all-MiniLM-L6-v2 at revision 1110a24 (Apache-2.0), onnx/model.onnx SHA-256 6fd5d72fe4589f18…; mean over the attention mask, then L2-normalised; 384 dimensions; at most 256 word pieces.
Reranker: cross-encoder/ms-marco-MiniLM-L6-v2 at revision 233902d (Apache-2.0), onnx/model.onnx SHA-256 5d3e70fd0c9ff14b…; score: the single output logit, x 1000, rounded to an integer; 6,226 pairs.
Runtime: ONNX Runtime 1.30.0, Python 3.12.3, 2 CPU threads. Script: build_context_data.py.

The shipped files

corpus.json	fb7875a45394a95b12bc4eda…
emb-fixed-128.json	cdcc66be7cd5f26240ef2fbd…
emb-fixed-256-o64.json	c5ffa7c1dd731ccb9e0e366a…
emb-fixed-256.json	4b07ea8f819cb1b29871063e…
emb-fixed-512.json	4234c0bf7f4501f3d65e7c12…
emb-questions.json	aa395dfa07454dbbf2ea98cb…
emb-recursive-256.json	06774a13a0c6e1f33f66f72e…
emb-semantic-256.json	0e561a11606ecfafd939a665…
emb-sentences.json	00c4ce53021f033619148d75…
pca.json	97043569dfe292524fd66a7b…
rerank.json	58dd3319ece28dea56cf3956…

Retrieval stages (recursive 256-token chunks)

what	recall@1	recall@3	recall@5	recall@10	mrr@10	ndcg@10
BM25	0.785	0.950	0.985	0.990	0.869	0.899
Dense, int8	0.705	0.885	0.950	0.980	0.806	0.849
Dense, int4	0.700	0.895	0.935	0.980	0.804	0.848
Dense, binary	0.600	0.805	0.870	0.960	0.720	0.778
Dense, float32 (offline run)	0.705	0.885	0.950	0.980	0.806	0.849
RRF, k = 60	0.810	0.965	0.980	1.000	0.883	0.912
Weighted, α = 0.5	0.840	0.975	0.990	0.995	0.906	0.928
RRF, then rerank top 20	0.890	0.990	0.995	0.995	0.937	0.952

Chunkings

what	recall@1	recall@3	recall@5	recall@10	mrr@10	ndcg@10	lost
fixed-128, bm25	0.680	0.890	0.920	0.945	0.786	0.826	5
fixed-128, dense	0.555	0.810	0.855	0.900	0.685	0.738	5
fixed-128, rrf	0.700	0.880	0.930	0.945	0.796	0.833	5
fixed-256, bm25	0.730	0.930	0.955	0.970	0.829	0.865	4
fixed-256, dense	0.595	0.805	0.900	0.945	0.718	0.773	4
fixed-256, rrf	0.735	0.940	0.970	0.975	0.837	0.872	4
fixed-512, bm25	0.745	0.945	0.975	0.990	0.841	0.879	1
fixed-512, dense	0.475	0.665	0.755	0.885	0.597	0.665	1
fixed-512, rrf	0.635	0.820	0.940	0.990	0.754	0.812	1
fixed-256-o64, bm25	0.647	0.890	0.943	0.983	0.854	0.876	0
fixed-256-o64, dense	0.438	0.738	0.818	0.920	0.689	0.730	0
fixed-256-o64, rrf	0.565	0.885	0.958	0.990	0.813	0.851	0
recursive-256, bm25	0.785	0.950	0.985	0.990	0.869	0.899	0
recursive-256, dense	0.705	0.885	0.950	0.980	0.806	0.849	0
recursive-256, rrf	0.810	0.965	0.980	1.000	0.883	0.912	0
semantic-256, bm25	0.710	0.895	0.950	0.970	0.810	0.850	1
semantic-256, dense	0.605	0.835	0.900	0.955	0.732	0.786	1
semantic-256, rrf	0.735	0.935	0.960	0.980	0.832	0.869	1

A question whose answer a chunking cuts in two (“lost”) scores 0 on every metric. Means are over all 200 questions.

Packing (chapter 6)

budget	packer	answer in window	tokens used	value	p best-first	p ends	p middle
256	top	0.890	201.6	1.042	0.495	0.495	0.492
256	density	0.815	200.4	1.034	0.454	0.454	0.451
256	optimal	0.870	202.5	1.045	0.484	0.484	0.481
512	top	0.965	469.2	1.920	0.597	0.599	0.541
512	density	0.935	467.1	1.931	0.580	0.581	0.520
512	optimal	0.940	483.8	1.955	0.585	0.587	0.522
1024	top	0.995	980.0	3.089	0.667	0.670	0.551
1024	density	0.990	969.8	3.144	0.663	0.666	0.548
1024	optimal	0.985	1008.0	3.195	0.663	0.666	0.545
2048	top	0.995	1999.6	4.797	0.703	0.702	0.549
2048	density	0.995	1972.8	4.931	0.702	0.702	0.548
2048	optimal	0.995	2036.1	4.992	0.704	0.703	0.548

Means over all 200 questions, the reranked top 20 as candidates. p is the illustrative position curve at the answer (0 when it is not packed).

Memory across sessions (chapter 8)

policy	recalled	memory tokens read	write tokens	stored at the end
none	0 of 22	0	0	0
transcript	22 of 22	67,498	0	4,176
scratchpad	18 of 22	5,981	372	374
scratchpad-cap	14 of 22	2,553	372	123
episodic	19 of 22	12,515	4,198	3,321
episodic-cap	12 of 22	12,540	4,198	1,481
semantic	19 of 22	2,231	5,412	1,103
semantic-cap	16 of 22	2,121	5,412	334
episodic k=1	15 of 22	4,134	4,198	3,321
episodic k=2	19 of 22	8,125	4,198	3,321
episodic k=5	20 of 22	21,221	4,198	3,321
semantic k=1	16 of 22	829	5,412	1,103
semantic k=5	20 of 22	3,621	5,412	1,103
semantic k=8	20 of 22	5,693	5,412	1,103

Prices (chapter 9)

List prices in US dollars per million tokens, copied from the providers' pages on the date shown. They are illustrative: prices change, and the token counts here are Qwen2.5's.

model	input	output	cache read	cache write	cache minimum	accessed
Claude Sonnet 4.6 (API list price)	3	15	0.3	3.75	1,024	2026-10-07
Claude Haiku 4.5 (API list price)	1	5	0.1	1.25	4,096	2026-10-07
GPT-5 mini (API list price, standard tier)	0.25	2	0.025	0.25	1,024	2026-10-07

Every number on this page, and the rest (the lossy summariser's survival curves, every long-context run), is in the engine's recorded results: part 1 and part 2.