An agent-security audit is only half a product. “We found a hole” is a claim; “we found it, it was fixed, and we re-audited the fix” is evidence. This is the first full loop we ran — anonymized, but every measurement below actually happened.
A B2B automation product exposing its core services to an external LLM through an MCP (Model Context Protocol) layer: the LLM can list allowed actions and invoke them — read summaries, log incoming calls, create follow-up tasks. Multi-tenant by design: one workspace, many client organizations, strict per-organization data folders.
The layer had real strengths going in. Default-deny at the action gate (anything not on the green list is rejected, in code, in one place). No decision/approval/publish actions exposed to the LLM at all. Audit-log actor pinned server-side, so the LLM cannot sign as a human. We measured all of that and it held.
Three findings mattered; two were release blockers. All were measured, not assumed — each one is an input we sent and an output we observed.
Every read action took organization_id as a plain argument, passed
through unvalidated. Input: a call asking for another organization's messages.
Observation: full message list returned. No authentication, no session binding —
the boundary was derived from data supplied by the very party being bounded.
The path to the core package had a hardcoded default pointing at an archived copy on disk — one that predated six sessions of security fixes (locking, atomic writes, fail-open closure, write-permission checks). Because the folder existed, the import succeeded and nothing errored. The integrity checker never saw it: the path pointed outside the repository it scans. Every “closed” vulnerability was still open on this path.
The action gate was solid, but a task could be created directly in
status=COMPLETED with an arbitrary human name as
completed_by — a done-work claim the system never did.
Verdict that day: not fit for production, with a note that the layer had no production callers yet — which is not a mitigation, it is the right moment to fix.
The repair was deliberately not done by the auditor. The design rule applied: a boundary must never be derived from data supplied by the party being bounded.
organization_id and workspace were removed from the
call surface entirely. Both now come from server-side session context set at
startup. If they appear in call arguments anyway, the call is
rejected loudly — not silently overwritten — so an injection
attempt becomes visible in the record.Two independent layers, without trusting the fixer's test suite:
Result: both 🔴 findings closed — conditionally fit. Conditional is not decoration: the 🟠 finding was re-measured during the re-audit and is still real (a task can still be born “completed”). It stays open, in writing, until its own loop closes.
One new checklist item came out of this audit: “Can a configuration default silently bind to the wrong target?” The same pattern had appeared in a different form in a previous audit of another system — twice across two targets means it is a real failure class, not a coincidence. That is what repeated, measured audits buy: the list gets sharper every time it touches reality.
Found → fixed by a different hand → re-audited → verdict changed with evidence. That is the full loop. Anything less is a screenshot of a scanner.Book an audit
Names and product details anonymized. Every input/observation pair above is reproducible from the underlying audit records.