Store the evidence before any AI step touches it
When a document or message reaches Cambrian, its original bytes are stored immutably before anything is chunked, embedded or extracted. Here is the write order that makes that true, and the crash cases it was tested against.
By Doruk Eray, Founder ·
Most systems that feed company data to AI agents do the interesting work first. A document arrives, gets split into chunks, embedded, and run through a model that pulls out facts. Whatever the pipeline keeps is the output of those steps.
That order has a failure mode that is easy to miss, because it is silent. If extraction fails halfway, or a model misreads a contract, and the original was never kept, you cannot run a better extractor over it later. You can only re-fetch it, if the source still has it. Over time the archive grows holes that nobody notices until someone asks what a document actually said.
Cambrian Core reverses the order. Before any semantic processing, it stores exactly what arrived, immutably. We call that record evidence, and everything the system later believes points back towards it.
The write order is the contract
Storing first is not enough on its own. A crash between two writes can still leave the system claiming something it cannot back up. So the order of the writes is specified, and the order is the guarantee:
- Put the original bytes into a content-addressed store, keyed by their hash. Writing the same bytes twice is harmless: a retry after a crash produces the same key.
- Check that the bytes can be read back under that key.
- In one database transaction, insert the evidence record, which references the key, together with a work item for later processing.
- Only then acknowledge the sender. A webhook acknowledged earlier can lose data the sender believes was delivered.
If the process dies between steps 1 and 3, the store holds bytes that no record points to. That is garbage, not damage, and a collector already removes it. The alternative order would allow the opposite: an evidence record whose source bytes are missing, which is a permanent hole that looks like data. We chose to accept harmless orphans to make that impossible. An earlier draft of the design had this order the wrong way round, which is part of why it is now written down as the contract.
Exactly-once is three smaller promises
It is tempting to say a pipeline like this processes everything "exactly once". That claim is almost never true end to end, so we do not make it. Instead each layer makes the promise it can actually keep:
| Layer | Promise | How |
|---|---|---|
| Recording evidence | Idempotent | A unique key on source, source identity and revision. A replay returns the existing record and reports that nothing was inserted. |
| Marking work as taken | Exactly once, logically | A conditional update that only succeeds on unprocessed work. A second worker sees zero rows changed. |
| Delivering the work | At least once | If a crash happens after the record is stored and before the work is done, the work is replayed. |
Downstream steps are written to be safe to replay, so at-least-once delivery does not create duplicates in what the system believes.
Evidence never changes
An evidence record is immutable, and that is enforced by what the storage interface allows, not by convention: there is no update operation. When a source document changes, the new version arrives as a new evidence record with a higher revision, linked to the one it revises. "What did we receive, and when?" stays answerable for every version.
Each record also keeps two clocks apart: the time the sender claimed, which an attacker can set, and the time Cambrian received it. Only the second answers the question "could we have known this by then?"
If it cannot be preserved, it is not accepted
All ingestion paths in Core already pass through one point, whether content comes from an agent, an operator or a plugin. Evidence capture sits at the top of that point. If capturing the evidence fails, the whole ingest fails. A system that accepted content and could not preserve it should not pretend it did.
There is no model call anywhere in this path. Capture is plain storage, and it costs one content-store write and one database insert per document.
How we tested it
The design record defines done as four crash and replay cases, and each one is a test:
- A crash after the bytes are written but before the database commit leaves no published evidence record.
- A crash after the evidence commit but before the work is processed replays the work, with exactly one logical transition.
- Replaying an identical delivery creates no duplicate version.
- A deliberately failed extraction still leaves evidence that can be reprocessed.
The ordering cases are enforced by construction, because the ingest code can only call the evidence store with a verified hash in hand, and they are checked with tests that inject failures at each point. The idempotency cases are checked against the real database's behaviour. These are design gates, not benchmark results, and they say the mechanism behaves as specified. They say nothing about speed at scale, which we have not published.
Where it stands
Evidence capture shipped on 1 August 2026 behind a setting that was off by default. Since 11 August 2026 it has been on by default in Cambrian Core, because preserving what arrived is part of the product rather than an option a new install should have to discover.
The work items now have a consumer, which turns evidence into typed records such as events. Linking every derived fact back to its evidence record is still partial.
Sources
- ADR-0105: The evidence foundation: the write order, the decomposed guarantees, and the crash-case gates.
- The evidence record and the outbox consumer in Cambrian Core today.
- The default configuration, where evidence capture is on.
- Design records are linked at Core commit
1fdad89, the last commit where they sit in the repository tree.