← back to the archiveCover illustration for “Agent governance is compiling into the language”
ESSAYday 64·5w ago·by Andy Padia

Agent governance is compiling into the language

One essay says agent governance is the layer nobody built. The same weekend, NVIDIA's NOOA makes prompts docstrings and contracts type annotations. Most of it is version control.

Two things landed the same weekend, arguing past each other.

An essay went out on August 3 titled, in effect, the governance layer nobody built yet — the case being that agent loops need an external control plane sitting above them. It illustrates the need with two cost incidents: $47,000 over eleven days, and $4,200 across 240 retries in three hours. No company, no date, no source for either.

Ten days earlier, NVIDIA's labs group had submitted NOOA (arXiv 2607.20709, fifteen authors), which formalises agents as ordinary typed Python objects. The docstrings are the prompts. The type annotations are the contracts. It reports 82.2% on SWE-bench Verified with GPT-5.5 at xhigh reasoning, 73.0% on Terminal-Bench 2.0, and 97.9% — 4,309 of 4,400 — on its own capability tests.

One says governance is a missing layer outside the loop. The other quietly moves the loop's definition into reviewable source code.

What "governance" mostly turns out to mean

Ask an enterprise what agent governance means in practice and you get a fairly consistent list. Who changed the prompt. What is this thing allowed to touch. Which version ran when the incident happened. Can we revert it. Who approved the change.

Read that list again. Every item is version-control machinery. Authorship, permissions, versioning, rollback, review — these are solved problems with thirty years of tooling, and enterprises already have the org muscle for them.

If the prompt is a docstring and the tool contract is a type annotation, all of it comes free. The prompt change shows up in a diff. The capability surface is a type signature a reviewer can read. The version that ran is a commit. Reverting is a revert. You do not need a new control plane to answer those questions; you need the agent's behaviour to live in the artifact your organisation already governs.

That is the arguable claim, and I will put it plainly: most of what is being sold as agent governance is repository hygiene wearing a new label, and language-level agents deliver it as a side effect.

Two approaches to the same question set. On the left, an external governance layer proposed above the agent loop. On the right, the same questions — who changed the prompt, what may it touch, which version ran, can we revert — answered by ordinary source-control machinery once the prompt is a docstring and the contract is a type annotation. A separate box marks the residue the language cannot cover: outer-loop kill authority and spend ceilings.

The part the language cannot do

I want to hold the counter-argument honestly, because the essay is not wrong about everything.

Kill authority cannot live inside the object being governed. A spend ceiling, an organisation-wide stop, a circuit breaker that fires when an agent is looping expensively — those have to sit outside the loop by construction, because an agent that can disable its own limit is not limited. Both of the essay's cost incidents, if real, are exactly this failure mode: nothing outside the loop was counting.

So the two positions attack different surfaces. The language covers authorship, permissions, versioning, reversion — the audit surface. The outer layer covers termination and spend — the safety surface. My claim is not that the outer layer is unnecessary. It is that the discourse has the proportions backwards: the audit surface is most of what enterprises actually ask about, and it is already solved, while the genuinely unsolved part is narrow and mostly about counting money.

The evidence contrast is itself the story

There is a second thing worth noticing, and it is uncomfortable for the side I am agreeing with less.

The governance essay runs on two anonymous incidents with no company, date, or source — folklore-grade evidence, by the source's own design. The language-level artifact ships with a public repository, a CC-BY paper, and per-configuration benchmark tables.

That asymmetry should influence how much weight each gets in a budget conversation, and usually it does the opposite, because the essay is readable and the paper is not. I have been guilty of this myself: a well-argued post travels through an organisation far faster than a benchmark table, and the post is what ends up quoted in the deck.

NOOA's numbers deserve their own scepticism, mind. They are vendor self-reported with no independent replication, the harness comparison fairness is not something I can assess from outside, and its claim to be the first combination of six ideas is not verifiable. Its being better-evidenced than an anonymous anecdote is a low bar, cleared — not a guarantee.

What I would actually do on Monday

At Trigent the useful move has been to stop treating agent governance as a procurement question and start auditing where the behaviour is defined. If the prompt lives in a database row edited through an admin console, no tooling will give you real governance — you have put the most consequential text in your system somewhere nobody reviews. If it lives in a file, in the repository, behind the same review your code gets, you already have most of the answer.

That reframing changes what gets bought. It usually turns a platform evaluation into a much smaller piece of work: move prompts and tool definitions into source, then add exactly one outer-loop control — a spend ceiling with a hard stop — because that is the piece the repository genuinely cannot provide.

Put the prompt in the repo and most of your governance question answers itself — then spend the budget on the one control the language cannot give you: the kill switch.

#agents#governance#engineering#harnesses#evidence
← older drop
US AI compliance dates move in both directions
newer drop →
Guardrail benchmarks are graded before the attacker moves

related drops

explore all 128 drops →
← back to the archiveday 105