The Question to Ask Any AI Agent: What Can It Break Before Anyone Notices?

The short answer

Most discussion of AI agent safety is about whether the model will do something it should not. That is a real question, but it is not the one that decides how much damage an agent can do to you.

The deciding question is blast radius: the set of things an agent can change, using your credentials, while nobody is watching. A perfectly-aligned agent with unrestricted write access to your email, your repositories and your bank is more dangerous than a mediocre one that can only read. Capability and risk are not the same axis, and agents are the first software category where people routinely confuse them.

This is a practical framing, and it gives you a short list of questions worth putting to any agent you are considering — including ours.

Why agents break the usual security model

Conventional software has a comforting property: it does what it was written to do. A bug is a deviation from an intended path, the paths are finite, and you can enumerate them.

An agent's whole value proposition is that the path is not enumerated in advance. You state an outcome; it works out the steps. That means the set of actions it might take is not knowable at design time — only the set of actions it can take, which is defined by the permissions you gave it.

Three consequences follow, and they compound:

Permissions are the only real boundary. You cannot reason about what the agent will do, so the honest security model is to assume it might do anything within its permission set. This is uncomfortable but clarifying: it turns a vague question about model behaviour into a concrete question about access scopes.

Time without supervision multiplies everything. A wrong action a human sees immediately is an annoyance. The same action at 3am, followed by nine more built on its result, is an incident. Background execution is the single biggest risk multiplier in agent design, and it is also the entire point of a desktop AI agent. You do not get one without the other.

Reversibility is not evenly distributed. Drafting a document and sending it are adjacent steps that differ by orders of magnitude in consequence. Agents reason in natural language, where "send the summary to the team" and "write the summary" sit a few tokens apart. The severity gap is invisible in the representation the agent is thinking in.

Prompt injection is a permissions problem wearing a costume

The attack everyone has now heard of: an agent reads a web page, an email or a document that contains instructions, and follows them.

It is worth being clear about why this is structurally hard rather than a bug someone forgot to fix. A language model receives one stream of tokens. Your instruction and the content it retrieved arrive in the same channel, in the same format, with no cryptographic distinction between them. Every mitigation — delimiters, system-prompt hardening, classifier layers, provenance tagging — raises the cost of an attack. None of them make the channel trustworthy, because there is no channel separation to enforce.

Which means the durable defence is not "stop the injection" but "make a successful injection cheap". If the agent that read the hostile page cannot send mail, cannot push code and cannot spend money, a successful injection produces a wrong answer rather than an incident.

That is the argument for scoping by action rather than by trust in the model. It is also why an agent's web-browsing component and its messaging component should not share a permission set, even though it is far more convenient if they do. Vyra's autonomous web agent is the component most exposed to untrusted content, and the design principle is that reading the open web should not, by itself, unlock anything that writes.

Classifying actions by what you cannot undo

The practical control that does most of the work is boring: sort actions by reversibility and gate on that, not on confidence.

A rough hierarchy:

Reversible and private. Reading a file, running a search, drafting something. If it is wrong, you delete it. Agents should do these freely; requiring confirmation here trains you to click "yes" without reading, which is how confirmation dialogs stop working.

Reversible but visible. Renaming a file, moving a calendar event, editing a shared document. Undoable, but someone else may see the intermediate state. Worth logging clearly; usually not worth blocking.

Irreversible and private. Deleting files, overwriting data, force-pushing. Recoverable if you have backups, catastrophic if you do not.

Irreversible and public. Sending a message, publishing, spending money, changing permissions. You cannot unsend. These should always require a human, regardless of how confident the agent is.

The important design point: the gate should key on reversibility, not on the agent's confidence. Confidence is a property of the model's internal state and correlates poorly with correctness — a confidently wrong agent is the exact case you need the gate for. Reversibility is a property of the action, is knowable in advance, and does not depend on the model being right about anything.

Vyra's metacognition layer does irreversible-action detection for this reason, and the human-in-the-loop write-up goes into how the thresholds are set and why calibration is the hard part.

Where the data goes is a separate question

Permission scoping controls what an agent can do. It says nothing about where what it knows ends up.

An agent with persistent memory accumulates something more sensitive than any individual document: a structured picture of your working life, your collaborators, your commitments. Two questions worth asking about any such system:

1. Where does the memory live? On your device, or on a server belonging to the vendor? This determines who can be compelled to produce it, what a breach of the vendor exposes, and what happens if the company is acquired or shuts down — a scenario the Rewind shutdown made concrete for a lot of people.
2. What leaves the device per request? Even with local memory, cloud models mean context is transmitted to run inference. "Local memory" and "local inference" are different claims and are frequently blurred together.

Vyra holds memory on-device and routes through five model tiers with a local Ollama tier at the floor, so some work completes without anything leaving the machine. Being precise: the higher tiers are cloud models, and using them means sending context. That is a real trade, not a marketing footnote, and it is covered properly in local vs cloud models.

Questions worth asking any agent vendor

A checklist you can use on us or anyone else. If a vendor cannot answer these crisply, that is itself the answer:

• What credentials does it hold, and can I scope them per capability rather than all-or-nothing?
• Which actions are classified irreversible, and what specifically triggers a confirmation?
• What runs while I am away, and can I see a log of what it did?
• If it reads a hostile web page, what is the worst it can do afterwards?
• Where is my memory stored, and what is transmitted per request?
• How do I revoke everything at once, quickly?

The last one matters more than it sounds. Every agent should have an obvious, fast kill switch, because the first time you need one you will be in a hurry.

Common questions about AI agent security

Is prompt injection solved yet?

No, and it is unlikely to be solved in the sense people mean. The instruction and the retrieved content arrive in one undifferentiated token stream, so there is no boundary to enforce. Defences raise the cost of an attack; they do not eliminate the class. The reliable mitigation is limiting what a compromised agent can do, not trying to guarantee it cannot be compromised.

Are local models safer than cloud models for agents?

Safer in one specific way: nothing is transmitted, so a cloud vendor breach cannot expose your context. Not safer in others — a local model is generally weaker, and a weaker model reasoning about which actions are safe is a worse safety component. Local inference improves confidentiality, not judgement. Conflating those is a common error.

Should an agent ever act without asking?

Yes, or it is not an agent — an assistant that confirms everything is a slower chat window. The useful line is reversibility. Let it act freely where a mistake costs you a deletion, and require a human where a mistake cannot be taken back. Confirming everything is not caution; it is confirmation fatigue, which produces reflexive approval and is measurably worse than a well-placed gate.

What is the single most important control?

Scoped credentials. Almost every serious agent incident reduces to the agent having had access it did not need for the task at hand. Everything else — injection defences, confirmation gates, audit logs — is mitigation layered on top of an access decision that was made too generously at setup time.

---

Vyra runs a metacognition layer with irreversible-action detection, holds memory on-device, and includes a local model tier so some work never leaves the machine. It is in closed alpha, and the confirmation thresholds are exactly the thing that only calibrates against real use — if you try it and it asks too often or too rarely, that is the most useful feedback we can get. Join the waitlist.