What I am building: an open cybersecurity agent

I am building a cybersecurity agent: a model trained for security work, and the harness it runs inside. Both will be released open source under Prisma, the small open-source organisation I run.

I am writing this before it is finished, which is the uncomfortable way round. But a plan published in advance is checkable, and a plan published afterwards is a press release.

Why this problem

Security is one of the few domains where an agent’s core weakness is actually survivable, provided the harness is built correctly.

The weakness is that models produce confident output that is sometimes wrong. In most applications that is a silent failure — you get a plausible answer and no signal that it is bad. In security testing, a lot of claims are cheap to check. Either the request reached the endpoint or it did not. Either the parser accepted the malformed input or it rejected it. Either the port is open or it is closed. The ground truth is right there, in the response.

That changes the design. The model does not need to be right. It needs to be right often enough to be worth the compute, and the harness needs to verify every claim before it counts as a finding. A security agent that reports what it can demonstrate is useful. One that reports what it believes is a liability generator.

What I learned building the wrong version first

The first serious agent system I built was designed to run unattended. It planned, it acted, it moved on. In a demo it was genuinely impressive.

In real use it produced a specific kind of garbage. It would misread one intermediate result, carry that misreading forward as established fact, and keep working with total confidence for another ten steps. The output arrived complete and well-structured and was built on something that had never been true. Nothing errored. That was the problem — an exception would have told me something.

I deleted the orchestration layer and rebuilt it around a different rule: no step’s output is treated as fact until something outside the model confirms it. The result was less autonomous and dramatically more useful. Every belief I hold about agents comes from that rebuild, not from reading takes online.

(Note: this section is a draft based on our conversation — check that it matches what actually happened before you publish.)

The design

Evidence, not opinion. Every finding carries a reproduction: the exact request, the exact response, the conditions. If it cannot be reproduced by the harness, it does not get reported. This kills the false-positive flood that makes most automated security tooling ignorable.

A scope that is enforced, not requested. Targets are declared up front and the harness refuses anything outside them at the network layer. Scope written in a prompt is a suggestion. Scope enforced in the runtime is a boundary.

Small models, run locally. I have spent a long time on local inference with llama.cpp and my own model and provider configuration. Security work involves data that should not leave the machine, and an agent that requires an API call to a third party for every step is unusable in exactly the environments that most need it.

A full audit trail. Every action logged, every decision attributable, the whole run reconstructable afterwards. This is not a compliance checkbox. It is how you debug an agent, and it is the only way anyone can trust a report they did not generate themselves.

The obvious objection

An open model that is good at finding vulnerabilities is also a model that helps someone exploit them. I do not have a clean answer, and anyone who says the question is simple is not being serious.

My position: the tooling this replaces already exists and is already public. The asymmetry in security is not access to tools, it is attention — defenders have to cover everything, attackers only need one thing. Cheap, thorough, automated testing of your own systems shifts that asymmetry towards the defender, and it shifts it most for the people who currently get no security review at all because they cannot afford one. Keeping the capability closed does not remove it. It concentrates it in the hands of the people who can pay.

I release it under a scope-enforced harness for that reason, and I will keep arguing about it in public.

On the regulation

The thing that frustrates me most is not the technology. It is how long it takes to be allowed to do anything with it in Europe.

I am not arguing against rules for AI. Security tooling in particular should have rules. But there is a difference between rules and process, and what we have a lot of is process: approvals that take longer than the development cycle they govern, requirements written for a technology two generations back, and a compliance surface so large that only organisations with a legal department can cross it. The intent is to constrain the largest actors. The effect is to filter out everyone else, which leaves the field to exactly the actors the rules were written for.

Meanwhile the work still needs doing, and it will get done somewhere. I would prefer that somewhere to be here, in the open, where people can read the code.

Status

Model and harness are in progress. Both ship open source when the evidence pipeline is trustworthy end to end, not before. I would rather be late than release a security tool that confidently reports things it cannot prove.

I will write up what breaks along the way, including the parts that make me look bad. Those are the useful posts.