Compaction and summarisation

A 36-question task compacted by a perfect and a lossy summariser: which facts survive n compactions, measured over 20 seeded runs, and what searching again costs.

Loading the animation…

Concept

Chapter 1 compacted a twelve-question task with a summariser that never forgot anything. This chapter makes the task three times longer, 36 questions, six from each article, and asks what survives when the summariser is not perfect.

With no budget the long task costs 233,502 input tokens and every fact stays in view. Truncating to a 1,500-token window keeps only 4 facts. A perfect summariser keeps 32 for 98,532 tokens, in 24 compactions; it is not all 36 even so, because by the end the summary itself outgrows the budget and its oldest lines are trimmed. At 2,000 tokens the summary fits and all 36 survive.

A real summariser drops things. The engine's lossy summariser rewrites the whole summary at every compaction and keeps each line with probability $1 - \ell$ , drawn from the shared seeded generator, so a fact summarised $n$ times survives with probability $(1 - \ell)^n$ . Over 20 seeded runs at $\ell = 0.25$ and a 1,500-token window, the measured share of facts left after one compaction is 0.746, after two 0.557, after five 0.251 (the model says 0.237). The agent answers 10.7 of 36 questions on average.

Which facts go is not random: the oldest. A fact learned early is summarised again at every later compaction, so the first sixth of the task is answered in 3% of runs and the last sixth in 89%. Compaction does not forget evenly; it forgets the beginning of the task. A bigger window means fewer compactions and so more survivors: at 2,000 tokens the same summariser leaves 15.2 facts.

The remedy is the one from chapter 1: notice the fact is gone and search for it again. With re-retrieval every one of the 36 questions is answered at every loss rate, but the agent pays for it: 149,206 tokens in one run at $\ell = 0.25$ , about twice what compaction alone spends (73,214). Re-retrieval needs the source to still be there; when it is not, the next chapter's memories are all there is.

Maths

Let a fact enter the summary at compaction $c_0$ and the task have $C$ compactions. If each compaction keeps it independently with probability $1 - \ell$ , it is still there at the end with probability $(1-\ell)^{C - c_0 + 1}$ : geometric in how early it was learned. The measured curve $S(n) = \prod_{j \le n} \text{kept}_j / \text{faced}_j$ pools every fact of every run by the number of compactions it has faced, which is exactly that model's estimate; the small departures are sampling noise over 672 first compactions.

Re-retrieval turns a lost fact into a cost instead of an error: one search and up to three reads, each paying for a window of up to $B$ tokens, so recall goes back to 1 while the input cost rises by roughly (lost facts) $\times$ (reads per fact) $\times B$ .

Code

The lossy summariser, cut from the vendored engine (windowRun): every line of the new summary, old and new, is kept with probability $1 - \ell$ .

if (summariser === "lossy") {
  const kept: number[] = [];
  for (const f of lines) {
    const keepIt = rng.random() >= loss;