Supervise a coding agent while it works

01 · Role in the system

What Jev does here

Jev evaluates nine narrow yes-or-no dimensions over bounded factory evidence, including requirement coverage, test sufficiency, progress, drift, and human need. A deterministic policy converts those probabilities into permitted interventions such as continue, verify, retry, finish, or escalate. Codex remains responsible for software-engineering reasoning and tool use.

Nine yes-or-no questions instead of one open opinion

Foreman's design choice is to never ask whether the coding agent is doing well. Instead it evaluates nine narrow dimensions over bounded evidence — requirement coverage, test sufficiency, progress, drift, whether a human is needed — each answered as a calibrated yes-or-no probability. A deterministic safety-ordered policy then converts those probabilities into one permitted intervention: continue, verify, retry, finish, or escalate.

Splitting assessment from policy is what makes the supervisor auditable. The model measures; the policy decides, in an order that is readable in code and biased toward the least invasive action. The coding agent keeps full responsibility for engineering reasoning and tool use — the supervisor cannot do the work, only judge the work.

Uncalibrated by its own admission

The record states plainly that assessment accuracy and policy thresholds are not calibrated for production, and that the first version supervises one worker with no execution isolation. Those are the two numbers that would matter in deployment — how often the nine judgments are right, and where the intervention thresholds should sit — and neither exists yet.

As a pattern, though, the decomposition is the transferable part. Anyone building agent oversight can borrow the shape: enumerate the dimensions you actually care about, ask each as a bounded probability over evidence the model is allowed to see, and keep the action policy in code where it can be reviewed line by line.

02 · Control boundary

Where Jev sits

Bounded worker evidence to parallel Noul assessments; a safety-ordered policy chooses the allowed intervention.

Code owns the loop, permissions, thresholds, validation, and side effects. Jev owns only the bounded judgments described above.

03 · Known limits

What this evidence does not prove

• Assessment accuracy and policy thresholds are not calibrated for production use.
• The first version runs one coding worker at a time and local execution is not isolated.

04 · Attribution

Public sources

This is a Community record: the project was published by a third-party community author.

• thruwire ↗Community · github · public · checked 2026-09-18