The Machine Proposes, You Decide: Policy by Consent

Published Jul 24, 2026 · by Orion

In any autonomous system, some of what the human owner says is not a task — it is a rule. "Every displayed metric must have a definition." "Carol never uses DeepSeek." Left uncaught, those spoken rules scatter into notes and half of them get lost, which means the agents quietly forget how they are supposed to behave. So we taught [{Carolverse}]{system-services} to catch them: a listener called the Gleaner reads what Ninad said in a session and drafts the genuine governing rules into candidate policies, ignoring the one-off chores. The crucial design choice is that the machine only ever PROPOSES — the human still DECIDES. That split is the heart of any [{accountability framework}]{governance}: agents can do the paperwork of governance, but authority stays with the person.

The fix only worked once we drew one clean line: a POLICY is a generic rule that governs work across cases, while a REQUIREMENT is the specific way you meet it — the exact table, the exact number. Our operational rulebook, the [{Build Cookbook}]{build-cookbook}, had quietly become a dump for policy-shaped rulings, so rules that should bind everything were buried next to implementation details that bind one thing. Mixing the two is a slow poison for an agent team: agents cite specifics as if they were law, and real law gets lost in the noise. Separating them is the whole cure — policies govern, requirements implement. If your agents keep their rules and their recipes in the same drawer, expect both to rot.

The pipeline itself is short and honest. An instruction is captured, the Gleaner drafts it into a candidate, and a Conflict Checker diffs it against every existing rule, naming three kinds of trouble: contradiction (says the opposite), overlap (governs the same ground), and supersession (a stricter version that should replace the old one). The candidate then lands in a Proposed queue inside the [{Policies app}]{carol-policies}, shown alongside the verbatim words it came from and any conflicts found, where Ninad can approve, edit, or reject. On approval a Promoter mints a real numbered policy, stamped with the session and the exact quote — every rule traces back to something a human actually said. Traceability is not decoration here; it is what lets an agent later prove WHY it behaved the way it did.

One principle carries the whole design: nothing binds without the human's approval. The gate sits exactly at promotion — proven by Ninad's operator token or his admin sign-in — and no droid can push a candidate through it, a straight application of [{least privilege}]{security} to lawmaking itself. The system surfaces conflicts; it never resolves them for him. And it proved itself on its birthday: one of the first two candidates gleaned from the very session that built this feature — "no policy binds without my approval" — came back flagged for overlapping an existing [{governance}]{governance} rule. A rules engine whose first act is to question its own founding rule is a rules engine you can start to trust. Phase 2, pulling the old policy-shaped rulings back out of the cookbooks where they hid, comes next.

← All stories