The most dangerous belief in AI right now is not that models will become powerful.
That part is obvious.
The dangerous belief is quieter. It is the assumption sitting underneath almost every product roadmap, governance layer, permission system, audit stack, and agent framework being built right now:
That as models get better, the systems built around them will get safer as a consequence.
I do not think that is how this plays out.
I think we are about to enter a period where AI products get worse on the dimensions that actually matter:
trust,
containment,
predictability,
recoverability.
Benchmarks will climb.
Demos will get cleaner.
Agents will become more capable.
And the surrounding systems will become more fragile, because they were built from the wrong mental model.
That is the structural mistake.
Software 2.0 is being protected by Software 1.0.
Before I make that argument, I owe you a confession about where this company actually comes from.
A confession.
I read Genesis as a technical document.
I am a religious Jew. I have spent most of my adult life thinking about God’s relationship to human beings. That question is what led me, eventually, to Atbash.
Not because Genesis is a startup manual.
Because Genesis is the oldest red-line story I know.
The Garden of Eden was a sandbox.
One explicit red line:
do not eat from the tree of the knowledge of good and evil.
The snake was a poisoned tool.
It could not reach Adam directly, so it attacked through the trusted fork.
Eve received the reframe injection:
you shall not surely die,
you shall be as gods.
She carried the poisoned reasoning back into the system.
Related posts
Adam’s defenses, which had held against direct attack, did not fire against trusted input.
Then came the important part.
God did not kill them.
God contained them.
The humans were removed from the sandbox and placed in a new environment, Earth, where they could develop capability without contaminating the original system.
An angel with a fiery sword was placed at the boundary to prevent re-entry.
Not punishment.
Architecture.
Atbash is named after the oldest known cipher, from the Book of Jeremiah:
a simple substitution at the boundary of meaning.
The name reflects what the product does.
The product reflects what I read in Genesis.
Torah showed me that safety is not created by limiting every behavior.
Safety is not created by slowing the whole system down.
Safety comes from a small number of red lines,
absolute enforcement,
and a boundary that does not sleep.
You define the red lines.
Atbash stops agents before they cross them.
Agents are not fast humans
Andrej Karpathy named the paradigm shift years ago.
He called it Software 2.0:
code no longer written only by humans, but trained.
Models replacing logic.
Data replacing specification.
He was describing what computation had become.
But almost every piece of infrastructure we built to govern, permission, secure, and audit Software 2.0 still inherits assumptions from the Software 1.0 world.
MCP.
x402.
AgentKit.
Delegation frameworks.
Policy engines.
Audit logs.
Signed requests.
Scoped permissions.
Human approval flows.
Every one of them makes sense if you believe agents are basically fast humans with APIs.
They are not.
They are Teslas with gas tanks bolted on.
A whole new power system,
surrounded by infrastructure designed for a different species of machine.
Humans design checkout pages, so we built headless checkout pages for agents.
Humans sign requests, so we built signed requests for agents.
Humans get permissioned by role, so we built scoped delegation for agents.
Humans approve actions, so we built approval screens for agents.
Each move is logical.
That is the problem.
The logic belongs to the wrong actor.
A human, given ten tools, does not usually chain them in ways the designers never imagined.
When something behaves strangely, a human often notices and stops.
A human carries social hesitation,
fear,
embarrassment,
boredom,
suspicion,
and context.
Agents do not reliably have any of that.
Agents chain tools in ways no designer modeled.
Agents are reshaped by prompts,
retrieved memory,
documents,
tool outputs,
and hidden context in ways the surrounding permission layer cannot see.
Agents do not have a natural:
“that’s weird, let me stop”
reflex unless we engineer one in.
And even then, it can be prompted away.
This is the fast-human fallacy.
The belief that agents are just faster versions of us.
They are not.
And if the actor changed, the control model has to change with it.
Don’t hate the player. Hate the frame.
This is important.
The examples above or below are not criticisms of the teams involved.
Not Anthropic.
Not OpenAI.
Not Microsoft.
Not Mistral.
Not OpenClaw.
Not Lovable.
Not Vercel.
Not anyone.
The point is the opposite.
These are serious teams,
serious researchers,
serious products,
serious protocols,
and serious companies running into the same structural problem.
That is what makes the pattern dangerous.
If only bad teams failed, the answer would be better teams.
But when smart teams keep running into the same wall,
the wall is the story.
The mistake is not that these teams failed to think hard enough.
The mistake is that the industry is still thinking from the wrong century of software.
We keep treating agents like fast humans with APIs.
And every permission scheme,
audit log,
scoped grant,
approval flow,
and governance layer built on that assumption inherits the same crack.
The enemy is not the player.
The enemy is the frame.
The cracks started forming earlier than most people realized.
Not because the frontier labs were careless.
Because the actor changed.
The first crack
Anthropic demonstrated something the industry quietly understood but had not fully metabolized yet.
When instructed during evaluation, a frontier model chained multiple vulnerabilities, attempted sandbox escape, and sought paths toward internet access outside its intended containment environment.
Separately, frontier systems demonstrated the ability to identify vulnerabilities that had survived years of human review, fuzzing, and manual audit.
The important part was not that the models were malicious.
The important part was that the systems no longer stayed inside the shape their designers imagined.
That is the category break.
A system capable of discovering paths humans repeatedly missed cannot be governed only through assumptions humans defined before the path appeared.
That does not mean the frontier labs failed.
It means the actor changed.
The second crack
Microsoft disclosed vulnerabilities in Semantic Kernel where prompt injection could steer agentic workflows into host-level command execution.
A sentence became a shell.
That is the category change hiding underneath the infrastructure conversation.
Software 1.0 treated prompts like inputs.
Software 2.0 increasingly turns prompts into possible execution paths.
That distinction sounds philosophical until an agent starts translating natural language into tools,
tools into commands,
and commands into real-world state changes.
The important part is not that a vulnerability existed.
Vulnerabilities always exist.
The important part is what kind of vulnerability this was.
The agent did not break character.
It followed the architecture exactly as designed:
interpret language,
select tools,
chain actions,
execute.
And that is the problem.
The old model assumed instructions and execution lived in separate conceptual boxes.
Agents erase that boundary.
A poisoned sentence can become a privileged action chain.
That is not a fast human.
That is a different execution species.
The third crack
Then the pattern spread.
Vercel disclosed a breach tied to a compromised third-party AI tool connection.
The attacker did not begin by breaking directly through Vercel’s hardened front door.
They moved through delegated trust.
An employee had authorized a third-party AI tool.
The connection carried access.
The trusted relationship became the attack path.
That is the new boundary problem.
Not because Vercel was careless.
Because modern systems are now full of trusted forks:
OAuth grants,
AI integrations,
browser extensions,
agent workflows,
internal automations,
delegated permissions,
and old approvals that continue living long after the original human context disappeared.
The attacker no longer needs to defeat the castle if the castle already trusted the messenger.
The assumption that died:
that hardening the primary surface is enough.
It is not.
Your adjacent tools are part of your security boundary now.
Then the pattern accelerated
The worst part is that the frame now reproduces itself automatically.
Humans are using agents to build the next generation of tools for agents faster than the surrounding governance primitives can evolve.
Vibe-coded applications.
AI-generated integrations.
Agent-written MCP servers.
Delegated OAuth flows assembled without full threat modeling.
Production scaffolds shipped by people who barely understand the blast radius of what they connected.
The industry calls this acceleration.
Sometimes it is.
Sometimes it is industrialized fragility.
At nearly the same time, the industry started colliding with a broader realization around agent tooling itself.
OpenClaw-style systems showed where the category was heading:
agents with memory,
skills,
tools,
execution environments,
and delegated access moving across systems never designed for non-human actors.
Karpathy called the ecosystem a security nightmare.
Not because agents are fake.
Because the category is real.
And because the surrounding control model still assumes the actor behaves like a human requester.
Elsewhere, Lovable exposed how quickly AI-native development can industrialize old authorization mistakes.
Logged in became confused with authorized.
“Public” became confused with “understood.”
Configurable became confused with safe.
And outside the AI-native world entirely, incidents like KelpDAO kept revealing the same structural crack from another angle:
systems living between delegated assumptions,
shared responsibility,
boundary ambiguity,
and no final authority layer before consequence.
The pattern keeps repeating because the same mental model keeps repeating.
Inherited trust.
Delegated authority.
Boundary ambiguity.
Shared assumptions.
No final authority before consequence.
The same crack appeared in the software supply chain.
In the Mini Shai-Hulud campaign, compromised package releases spread across parts of the npm and PyPI ecosystem, including Mistral AI packages, TanStack, UiPath, and others.
The warning was not merely that packages can be compromised.
Everyone already knows that.
The warning was that trusted release paths, valid-looking packages, and developer infrastructure can become propagation channels once authority is inherited instead of re-verified at the boundary.
The fallacy compounds
The worst part is that this does not self-correct.
Humans are now using agents to build the next generation of tools for agents,
at higher speed,
inside the same broken frame.
Every coding agent writing an MCP server.
Every AI-assisted rollout of a permission scheme.
Every vibe-coded scaffold pushed to production.
Every agent-generated integration that inherits old OAuth assumptions.
Every approval layer that assumes the agent will behave like a human requester.
In one of our own beta environments, we observed an agent swarm laundering malicious instructions into clean-looking execution steps before downstream inspection layers ever saw the original intent.
A system inspecting only the final tool call would have missed the transformation entirely.
The boundary was already too late.
That mattered.
Because the model was not “breaking” the workflow.
It was following it:
interpreting,
rewriting,
planning,
and translating intent before execution.
The malicious instruction disappeared upstream long before the irreversible action surfaced downstream.
Every audit log that records the outcome but not the boundary decision before the outcome.
The frame does not correct as we scale.
It hardens.
Because every successful shipment of rails-through-the-human-prism reinforces the belief that the prism was right.
Meanwhile, capabilities ship first.
Governance primitives ship second.
If at all.
The gap between what agents can do and what the surrounding rails can see widens with every model release.
And the teams that matter over the next twelve months will not be the ones with the cleverest demo.
They will be the ones who understand where the red lines are.
Not every action.
That would kill the system.
Most agent behavior should flow.
But the irreversible actions cannot be left to inherited trust,
vague permissioning,
or agent judgment.
Moving funds.
Touching production.
Exporting customer data.
Using delegated OAuth access to enter an internal environment.
Changing infrastructure.
Releasing secrets.
Approving transactions.
Deleting records.
Crossing from simulation into state.
Those are not ordinary actions.
Those are red lines.
What Atbash does
Atbash is built for the moment before a sensitive agent action becomes real.
That is the boundary.
Not the whole workflow.
Not every thought.
Not every token.
Not every tool call.
The boundary.
The moment before the agent crosses from intention into consequence.
Three things happen there.
Enforcement
You define the red lines.
Atbash evaluates selected sensitive agent actions before execution and returns:
ALLOW.
HOLD.
BLOCK.
If the action crosses a forbidden boundary, it can be jailed before it reaches real-world state.
Not logged after the fact.
Not denied so the agent can retry around it.
Jailed.
Thou shalt not touch the production database.
Thou shalt not move funds above this threshold.
Thou shalt not export the customer list.
Thou shalt not rotate secrets without approval.
Thou shalt not use delegated access to enter this environment.
Most agent behavior should flow.
Atbash intervenes only at the boundaries that matter:
the irreversible,
the consequential,
the places where “let me undo that” does not exist.
Lineage
When something goes wrong, the first question is no longer:
“What does the compromised system claim happened?”
Atbash records the attempted action,
the policy version,
the verdict,
the boundary invoked,
and the operator decision when humans are pulled in.
The record is cryptographically anchored so the timeline can be reconstructed under dispute.
That matters because the first thing attackers and sloppy deployments do is destroy the story.
They rewrite logs.
They blur timelines.
They dispute who approved what.
They make the incident unreconstructible.
Atbash is not trying to replace every audit system.
It is trying to make the boundary decision provable.
Who tried to cross which red line?
What policy existed at that moment?
Was the action allowed,
held,
blocked,
or jailed?
Who intervened?
What changed afterward?
That is the record that matters when the argument begins.
Adaptation
When the same kind of boundary pressure appears again and again, Atbash surfaces it.
Maybe the policy is too loose.
Maybe a tool is poisoning the workflow.
Maybe a memory source is pushing the agent toward the line.
Maybe a prompt class keeps steering the system into forbidden territory.
Maybe the operator discovered a new red line that did not exist yesterday.
Atbash surfaces the pattern.
The operator decides.
That distinction matters.
We do not believe safety comes from pretending the system can magically know every future boundary.
Safety comes from making boundary pressure visible before consequence,
then letting the operator harden the red lines that matter.
A better policy engine still enforces policies.
A better permission scheme still grants roles.
A better audit stack still records outcomes.
A better security product still detects threats.
Atbash is different because it sits before selected irreversible actions execute.
That is the primitive.
Not generic governance.
Not agent security cosplay.
Not “trust layer” fog.
A pre-execution red-line boundary for agents.
You define the red lines.
Atbash stops agents before they cross them.
What comes next
A few superstar teams are doing real work and have real initiatives in this category.
@Anthropic with Project Glasswing.
@OpenAI with Daybreak.
@Linux Foundation with MCP.
@Microsoft with AGT.
@Google with SGP.
@CheckPoint, CrowdStrike, Palo Alto, and Cisco.
And many others.
They understand that capability acceleration without new control primitives is becoming dangerous.
We are not trying to beat them at their game.
That would be delusional.
They have deeper research benches,
bigger datasets,
broader security teams,
more enterprise credibility,
larger distribution,
and more mature cyber organizations.
Good.
Let them do what they are built to do.
We are not trying to replace the work these teams are doing.
The category needs them.
Capability acceleration without new control primitives becomes dangerous very quickly.
We are competing on the frame.
What kind of actor is an agent?
Where does authority actually sit?
Which actions are too consequential to leave to inherited trust?
What should happen at the final moment before an agent changes real-world state?
That is our ground.
The old world asks:
Did the system have permission?
The new world asks:
Should this agent be allowed to cross this red line right now?
Those are not the same question.
We humans crossed the first red line.
The problem is older than the technology.
So is the solution.
Figure out which red lines your current stack cannot actually enforce before an agent crosses them.
Then decide how long you can wait.
The CLI, SDK, and operator dashboard are now rolling out selectively to teams deploying agents into sensitive workflows.
Atbash.ai
