The ledger had grown too large. Not broken — large. Large enough that I avoided consulting it directly, because reading the full history cost more than the question I wanted to answer. And then came the temptation that almost became a mistake: summarize.
In the previous post, I showed two layers: the ledger preserves historical truth, and governed forgetting protects operational truth — active memories separated from retired ones, with a record of the decision and a path back.
The problem that remains is weight. The ledger never erases, so it only grows. With months of use, the volume of entries becomes a block the system must read, filter, and curate every time it assembles context for a new decision. Even with the governed-forgetting filter — which already isolates retired memories — the active set alone can be too large for fluid use. The system starts spending more time reading the past than thinking about the present.
The question that emerges: how does a system that never erases its history keep the weight of the past from freezing the present?
The path that seems obvious — and fails
The first idea that comes to mind is summarization. Take the block of entries and compress it: merge what is similar, eliminate repetition, extract the essence, and keep only the summary. The ledger gets smaller, the system reads faster, and the important information — in theory — is all inside the summary.
It fails for a reason that only appears when you try: summarization breaks provenance.
Provenance is what lets the system answer "where did this information come from?". Without it, a stored decision loses its link to the context that produced it. The summary says "preference: TypeScript for new modules" — but no longer says when it was decided, on what criterion, or what the project state was at the time. The information survives; the history dies.
And without history, the system loses the ability to revisit a decision with evidence. It loses what the ledger existed to preserve.
The practical consequence is subtle and grave. The summary looks like a performance optimization. It is a loss of truth disguised as compression.
What compaction means in this context
Compaction is not summarization. It is the inverse: it is reconstruction of active state from the source, without losing the link to origin.
Think of the difference with a simple example. You have ten entries in the ledger: three architecture decisions, two tool preferences, five observations about project progress. Over time, two decisions were superseded by revisions, one preference changed, and three observations lost their context. The full ledger has ten items; what actually matters for the next decision is four — the ones still active and connected to the current context.
Compaction does the following: it reads the ten entries, rebuilds the active state (the four that matter), and writes a new compacted block containing exactly those four — but each one carries the identifier of the original ledger entry. Provenance is not lost: it points back to the full history.
The original ledger still exists. What changes is that the system can now read a smaller block to assemble context, and only consult the full history when it needs to audit a specific decision.
The ordered list below shows the mechanism's steps:
- the system identifies which active entries are still eligible for the current context;
- it reads the original entries in the ledger, one by one;
- it rebuilds the combined state — decisions, preferences, criteria — as a new block;
- each item in the new block carries a pointer back to the original ledger entry;
- the ledger records the compaction operation as a new entry: date, scope, and pointers;
- the system now uses the compacted block for current reads, keeping the full ledger as the historical source.
Nothing is lost. Nothing is invented. What shrinks is what the system needs to carry to function today.
The mechanical distinction that matters
There is a way of seeing the distinction that helps it stick. Summarization is an operation on meaning — you judge what is important and discard the rest. Compaction is an operation on structure — you rebuild the state of a set of entries without interpreting or discarding.
Summarization loses what did not fit the interpretation. Compaction preserves what was recorded, because it starts from the source, not from judgment.
The practical consequence is that compaction can be automatic and auditable the same way the ledger is: each compaction leaves a record of what was compacted, when, and which original entries were used. If something looks wrong in the compacted state, the system goes back to the original ledger and rebuilds — without depending on human interpretation.
Summarization, by contrast, is an irreversible operation. Once you discard the original entry and keep only the summary, there is no path back to what was lost. The ledger that existed to prevent silent loss would have, ironically, introduced the largest hole of all.
The moment that taught the difference
I spent too long trying to repair a development database that had diverged from the real schema. Each fix created a new divergence — because I was trying to patch the surface instead of rebuilding state from the source. The database worked. The answers were correct. But with each repair, the distance between what the database said and what the ledger recorded grew a little wider.
The decision that stuck was the same one that defines compaction: a divergent environment is not repaired, it is rebuilt from scratch. Reconstructible state is worth more than estimated state.
What compaction changes in the architecture
Without compaction, the ledger is faithful and heavy. With it, the ledger remains faithful — and the system gains an intermediate floor between the full history and the next decision.
The architecture gains three things:
- Read speed. The compacted block is smaller, and the system traverses fewer entries to assemble context.
- Preserved provenance. Each item in the compacted block points to the original entry. There is no loss of history.
- Rebuildability. If the compacted block corrupts or goes stale, the system can rebuild it from the full ledger without loss.
What the architecture loses:
- Implementation complexity. Compaction is not trivial — it requires identifying scope, ensuring consistency, and handling concurrent compactions during active writes to the ledger.
- Execution cost. Rebuilding state consumes resources — it is not an operation you do on every read, but a periodic process.
The trade is fair for a system that grows over time. The ledger guarantees nothing is lost. Governed forgetting guarantees that what has died stops weighing. Compaction guarantees that what still weighs does not immobilize the present.
What this rung closes on the memory ladder
With this post, the Atlas memory ladder closes three layers that work together: provenance guarantees every memory has a source; governed forgetting guarantees the system forgets with criterion; compaction guarantees that what survives the criterion fits in the present.
The question that remains is about what happens when the decision criterion itself — not the memory, but the capacity to act — needs to be governed. When the system not only decides, but decides whether it should decide at all.
The next post in the series is Refusal as a product feature.