The security hole our own benchmark found
Before giving agents access to Cambrian's knowledge layer, we wrote the test suite first and ran it against the system as it was. It failed four rows, and one of them was an access check that did not do what we thought it did. Here is what it found and how scope is enforced now.
By Doruk Eray, Founder ·
In early August, Cambrian's knowledge layer was finished in one sense and not in another. It had a typed query interface over evidence, facts and their history, and exactly one way in: a query lane for human operators in our commercial product. Agents could not use it at all.
Giving agents access to company knowledge is the point where access control stops being theoretical. So before writing any of that code, we wrote the test suite and ran it against the kernel as it stood, to record a baseline. I expected some rows to fail, because the features they test did not exist yet. I did not expect one of them.
The baseline: 12 of 16
The suite has 16 rows. Eleven price the query layer that already shipped: exact answers, beliefs over time, and refusals that name their reason. The rest describe the work we were about to do. The baseline run passed 12 and failed 4:
- Agents had no way in. No agent-facing call existed, scoped or otherwise. Expected.
- Answers did not cite the knowledge layer. Even a perfectly modelled fact reached an agent as one more ranked text chunk, with no record of where it came from. Expected.
- One operator query path was not protected the way we believed. The suite sent a request that should have been refused, and it was answered. Not expected.
The code's own comments said that path was protected. It was not.
Why it slipped through
The rule that protected operator requests covered the services it was written for. A service added later fell outside it. Nobody wrote "skip the check here". The gap came from a rule that protected things by where they happened to be, rather than by what they were for. The comments described the intent, and nobody had tested the behaviour.
The fix: declare the plane, do not rely on the name
Cambrian separates the agent plane, which agents call, from the operator plane, which people's tools call. We made that separation explicit at registration:
- A plugin service registered in the ordinary way is an operator-plane service, and every one of its methods requires operator credentials.
- A service that agents should reach must be registered through a separate call that marks it agent-facing. Its caller identity comes from the agent plane.
- The kernel collects both lists at registration and hands them to its interceptors. No service name is hard-coded anywhere in open source Core.
Scope: the caller says who it is, never what it may see
The new agent-facing path does not let the caller choose its own access. A plugin hands the kernel a principal, meaning who is asking, and the kernel looks up what that principal may read. An interface that accepted a filter instead would let a plugin pick its own scope, which is a bypass with extra steps.
Enforcement then depends on the shape of the answer, because each kind of row carries its classification differently:
| Query shape | How scope is enforced |
|---|---|
| Evidence | Filtered on the record's own classification. An unauthorised read returns nothing, not even the list of classifications. |
| Current beliefs and contradictions | Filtered before grouping, so a contradiction never shows one side because the other side was readable. |
| Points, history and events | Each row is traced to its evidence. A row whose evidence cannot be found is dropped. |
| Graph traversal | An edge is visible only if its evidence is, and traversal does not continue through a hidden edge. A forbidden edge is not a bridge. |
| Aggregates | Sums cannot be filtered after the fact, so the scope check is part of the database query itself. |
In open source Core, the default authoriser allows everything, so a deployment without access rules reads exactly as before. With the commercial access-policy plugin installed, the same checkpoint fails closed: no resolved scope means no rows. That asymmetry is deliberate. Memory fails open, so an agent still gets an answer when something optional breaks. Scope fails closed, so nobody reads what they should not.
After: 16 of 16
On 6 August 2026 the same suite passed 16 of 16. The request that had slipped through was refused, agents could query the knowledge layer within their scope, with no rows leaking outside it and none wrongly withheld, and answers carried citations. The eleven rows that price the existing query layer all still passed.
We extended the suite as we added features. When we added a natural-language question box, in which a model writes the query but never the answer, the suite grew to 21 rows, then 22 and 23 as early use turned up new cases. Each version passed in full.
These are results on our own suite. It is not an external security audit or a penetration test, and it only covers the cases we thought to write down.
What I took from it
The useful habit here was not cleverness. It was order: write the test for the feature before the feature, and run it against the system as it is. A baseline run is cheap, and it measures the code that is actually running, not the code everyone believes is running. Ours found a hole that the code's own comments said did not exist.
Sources
- ADR-0118: Substrate read-scope and agent retrieval: the baseline findings, the plane declaration, per-shape enforcement, the 16-row gate, and the addenda that grew it to 23 rows.
- ADR-0111: The typed query plane, the query layer being scoped.
- ADR-0085: The access-policy port, which defines the authoriser the scope check asks.
- The scoped query path in Cambrian Core today.
- Design records are linked at Core commit
1fdad89, the last commit where they sit in the repository tree.