The Jev Harness, Explained: What Coding Agents Would Look Like Without the KV Cache

The short answer

The Jev harness is a coding-agent design from notes by Diogo Almeida, founder of TypeSafe. It argues agents are shaped by KV cache economics: reusing a cached prompt prefix is cheap, editing earlier context is expensive. The fix stores state as typed chunks and lets Jev decide each turn what the model sees, which model and tool to use, and whether a command may run.

It is a design proposal, not a shipped product. The source is a 12-page PDF titled Jev Engineering for Coding Agents. It began circulating around 21 September 2026 and says it is "independently compiled" from Almeida's design notes and "not affiliated with or endorsed by TypeSafe." Some posts sharing it add speed and cost multipliers. Those numbers don't appear in the PDF, and we don't repeat them here.

What is Jev, and where does it sit in a coding agent?

Jev is TypeSafe AI's "System One" model. It does not write text. You give it state and a question, and it returns a typed answer with a calibrated probability. The answer is a choice from options you define, a score on a scale, or a noul (a yes/no probability). Almeida was previously at OpenAI working on post-training and alignment, and is a co-author of the InstructGPT paper, according to Latent Space's 21 September interview.

In the harness the PDF describes, Jev is not the model that writes code. It is the decision layer next to it. A frontier LLM writes, tools and code execute, and Jev answers small questions thousands of times per session:

Decision point	Question the harness asks Jev	Typed answer
Context	How visible should this chunk be for this query?	choice: hide / short / long / full
Cache	Reuse the cached prefix or rebuild?	noul + probability
Routing	Can this subtask leave the frontier model?	choice + cost estimate
Tools	Which tool fits this intent?	ranked choice, top-k
Permissions	Should this command run?	allow / ask / deny
Security	Which files will this task touch?	sensitivity score

Because each answer is typed, the harness can validate it, apply thresholds, and branch on it without parsing prose. That is the whole bet. The PDF puts it this way: "If an agent is just a loop, the loop is not where the leverage is."

Why does the KV cache shape how coding agents work?

The organizing question in the notes is: how would you design a coding agent if language models had no KV cache?

The KV cache is why agents are built as append-only transcripts. A model provider can reuse the computed state for a prompt prefix it has already seen. Anthropic, for example, bills cache hits at 10% of the base input price. Change anything early in the context and that saving is gone: everything after the change has to be processed again. The notes call this "the tyranny of the KV cache," and argue that it quietly sets six design choices that current agents inherit.

What are the six symptoms of the KV cache?

Symptom	Why it exists	What it costs
1. Routing fails	Handing work back to the large model means reprocessing context	Mixed routes cost more than staying on the frontier model
2. Tools crowd context	Every tool schema must sit in the system message	Tokens spent on irrelevant tools; weaker tool selection
3. Compaction	Assumes one shared state works for all future turns	Compresses before the next question is known, so it loses what matters later
4. Sub-agents are rare	Deciding what context to pass in and merge back is hard	Little automatic parallelism
5. Restarts	Long transcripts drift and corrupt	The good state gets thrown away with the bad
6. The batteries debate	Every built-in tool costs context permanently	A forced choice between ease of use and power

The compaction point is the easiest to agree with. A summary written before you know the next question will reliably drop something that question needed. Query-aware compression is much easier than generic compression, because you know what to keep. (We made a related argument about long-term memory in why an AI agent needs to forget on purpose.)

Does routing to a cheaper model actually save money?

Often, no. This is the most concrete claim in the notes, and the arithmetic holds up.

The intuitive plan is to let a frontier model plan, hand execution to a cheaper model, and bring the frontier model back to review. The notes price this with Opus at $5 input and $25 output per million tokens, and Sonnet at $3 and $15. Let X be context tokens, Y generated output tokens and Z tokens read during the work (file reads, command output), all in millions:

• Pure Opus: 25Y + 5Z. The context is assumed to be cached already.
• Opus → Sonnet → Opus: Sonnet loads the context (3X), generates (15Y) and reads (3Z), then Opus rereads what changed (5 × (Y + Z)). That totals 3X + 20Y + 8Z.

With the session shape the notes use (X = 0.65, Y = 0.12, Z = 0.23), pure Opus costs $4.15 and the routed path costs $6.19. The route meant to save money costs about 49% more.

We checked the arithmetic and reran it two ways:

Scenario	Pure frontier	Routed	Routed vs pure
Notes' prices (Opus $5/$25, Sonnet $3/$15)	$4.15	$6.19	+49%
Same, but charging Opus for cache reads of X at 10%	$4.48	$6.19	+38%
Current list prices (Opus 5.5 $4/$20, Sonnet 5 $2/$10)	$3.32	$4.36	+31%

The routed path loses in all three. Subtracting one formula from the other with the notes' prices shows when routing breaks even. The routed path only wins when output is large compared with everything read: Y has to exceed 0.6 × (X + Z). At current prices the threshold is Y > (X + Z) / 3. Coding sessions rarely look like that. They read far more than they write.

The notes don't conclude that routing is wrong. They conclude that routing priced per token, rather than per context rebuild, is wrong. Routing becomes viable only when the harness can hand the cheaper model a small, purpose-built context and merge the result back as a compact chunk, rather than a transcript the frontier model must reread.

Where do a coding agent's tokens actually go?

Mostly into reading, not writing. The PDF's own token-share table is labeled "an illustrative estimate", so treat it as a sketch. It puts file reading at 30–40% of processed tokens, codebase search at 10–18%, command output at 10–20%, and writing or editing code at only 4–10%.

The independent number behind it is stronger. Microsoft's FastContext team reports that in GPT-5.4 trajectories, "reading and searching account for 56.2% of all tool-use turns and 46.5% of the main agent's total tokens." Their repo-exploration sub-agent cut the main agent's token use by up to 60% in their tests. If that generalizes, the biggest efficiency gain in a coding agent is smarter retrieval, not a better model.

This matches what several September studies found about harness versus model. In HarnessTax, the harness moved success rates by only a few points but moved cost by up to 5x, largely because of how much context each harness carries into every turn.

What does the Jev harness propose instead?

One move: make state explicit and typed, and assemble context per query instead of accumulating it. Everything else follows from that.

Meta-attention: a visibility ladder for every chunk. For each query, Jev decides how visible every piece of state should be. Each tool input, tool output and piece of reasoning can be hidden, summarized briefly, summarized at length, or shown in full. A 2,400-line grep result can be twelve relevant hits for one question and invisible for the next, without ever being deleted. Jev also makes an explicit, cost-aware call on whether to reuse the cached prefix or rebuild. That keeps the useful part of compaction while fixing its main flaw.

Tiered tool disclosure. Instead of declaring every tool schema up front, the model sees one-line snippets for hundreds of capabilities. It gets the full schema only for tools it selects, and a manual only for one-off needs. The detail drops out of context when the task is done. The notes suggest this is part of why skills often beat raw tool lists and MCP servers. It would also end the batteries debate: a built-in that costs nothing until used can ship by default.

The harness as tool router. The model states what it wants in plain text. Jev picks the best tool, or the top few, and the harness constructs the arguments. A wrong argument type then becomes a validation error rather than a silent failure.

Programmable permissions. Instead of approving a command by name, permissions become queries. Deny anything touching ~/.ssh or .env*. Ask before writing outside the repo root. Read the contents of a shell or Python script before running it, and deny network egress unless the task is a deploy.

Conditional instructions. Today, AGENTS.md and CLAUDE.md load in full. The notes propose attaching sections to conditions: a style guide when the task touches *.tsx, a billing/GOTCHAS.md when working inside billing/. These fragments are pinned, so compaction can't summarize them away. Skills mean "do this now." Conditional instructions mean "keep this in memory whenever it applies."

Cheap sub-agents, with locks. Once building a small context is automatic, sub-agents stop being rare, and the harness inherits concurrency problems. The notes suggest shared state with locks, made tractable by typing every operation as a read or a write. Each subtask is also registered as a subgoal and deduplicated, so work already done or in flight is never launched twice.

What is security-aware routing?

Routing today weighs difficulty against cost. The notes add a third axis: trust. Some open-weight models served by low-cost providers are far cheaper than frontier APIs, but data sent to some of those endpoints may not stay private. The proposal is to estimate which files a subtask will touch, attach a policy to each file type, and route to match:

Files likely touched	Policy	Eligible models
Public docs, open-source dependencies	Open	Any, cheapest first
Application code	Standard	Vetted providers
Secrets, env files, infra config	Restricted	First-party frontier only
Proprietary research code	Custom	Excludes named vendors

This is the same instinct as asking what an agent can break before anyone notices: scope what the agent can reach before you decide how clever it is. Once routing is driven by policy, a team's vendor preferences become configuration rather than discipline.

Why do background tasks become cheap?

The notes list a pattern spreading through agent workflows: cross-model review, background eval generation, explain-like-I'm-five summaries, and a live progress page you can check from your phone. All of these run beside the main work and only read the codebase; they never write.

In a harness that is explicit about reads and writes, these tasks never contend for locks. And the expensive step, finding the files, symbols and diffs relevant to a change, can be done once and shared by every background task instead of repeated by each. Retrieval is the largest part of the token budget, so sharing it is the largest single saving the design offers.

What should you take from it?

Treat it as a well-argued design essay, not a benchmark. Three limits are worth stating:

1. Nothing here has been measured end to end. The routing arithmetic is sound, but the session shapes are illustrative, and the token-share table is an estimate. No Jev harness results are published in the PDF.
2. It is a synthesis, not a TypeSafe publication. It attributes its arguments to Almeida's notes "as provided to the compiler." Jev's own performance claims come from TypeSafe's self-run benchmarks (see our Jev explainer). Open-weight alternatives such as Laya could fill the same decision-layer role.
3. Prices move. The routing conclusion survives current prices, but the break-even point shifts whenever the price ratio between models changes.

What holds up without any benchmark is the framing: an agent's context window should be assembled on purpose, per question, rather than left to accumulate by accident.

Common questions about the Jev harness

Is the Jev harness an official TypeSafe product?

No. The 12-page PDF describes itself as an independent synthesis of design notes by Diogo Almeida, "not affiliated with or endorsed by TypeSafe." It is a design proposal, and it includes no released harness code or benchmark results.

Why does routing to a cheaper model cost more?

The cheaper model has to load the whole session context, and the frontier model then has to reprocess what the helper produced. With the notes' prices and session shape, that comes to $6.19 against $4.15 for staying on the frontier model. Routing only pays when the cheap model gets a small, purpose-built context.

What is meta-attention in a coding agent?

It is the proposal to decide, per query, how visible each chunk of agent state should be: hidden, a short summary, a long summary, or in full. Unlike compaction, which compresses once before the next question is known, it compresses after seeing the question. Nothing is deleted from state.

Do I need Jev to use these ideas?

No. Conditional instruction files, tiered tool disclosure, programmable permissions and routing by file sensitivity can each be built with ordinary code or any classifier. Jev's pitch is that typed, calibrated answers make these high-frequency decisions cheap and easy to validate.

Sources

• Jev Engineering for Coding Agents: The TypeSafe Founder's Blueprint for Building with Jev, independently compiled synthesis of design notes by Diogo Almeida, 12 pp., September 2026 (not a TypeSafe publication)
• Diogo Almeida's TypeSafe Coding Agent Notes (2026), explainx.ai, September 2026
• Jev: System One models for Prod, not God, with Diogo Almeida, Latent Space, 21 Sep 2026
• FastContext-1.0-4B-RL model card, Microsoft, Hugging Face, 2026
• Pricing, Claude API documentation, retrieved 26 Sep 2026

---

Vyra by Vyraagi is a desktop AI agent built around the same idea, that the harness and not the loop is where the leverage is. It's in closed alpha; if you want to try it, join the waitlist.