Teach
Five expert examples establish the food-aid coordinator’s expected behavior.
Evaluation-driven agent repair
Corrections that stay fixed.
ApprenticeOS turns every expert correction into a permanent evaluation, repairs the workflow, reruns the complete test suite, and refuses to deploy until every test passes.
Real evaluation engine Server-only OpenAI mode Deployment gate
food-aid / policy-v1
“My mother cannot eat peanuts. We are a family of four.”
Regression #30 added
Expert correction stored as an eval
Minimal policy repair
avoidance phrase + known allergen → high risk
Release gate open
Policy v2 · 0 regressions
The closed loop
From expert judgment to deployable behavior.
Not another chat interface. A focused control plane for a food-aid coordinator agent whose behavior becomes safer after every correction.
Test
A real policy gap misreads “cannot eat peanuts” as a preference.
Correct
One expert correction becomes permanent regression #30.
Repair
A minimal compound-signal rule updates policy v1 to v2.
Verify
All 30 cases rerun, including every earlier behavior.
Deploy
The release gate opens only at 30/30, then builds the Agent Pack.
Repeat
Every future correction strengthens the system’s evaluation memory.
Evaluation memory
The correction outlives the conversation and protects every later release.
Built for trust
The numbers are computed, not choreographed.
Every visible state is backed by policy files, evaluation records, and server-side actions. The demo remains fully reproducible without a paid API.
Corrections become test assets
The input, bad output, expert answer, reason, and timestamp are retained together.
Local evaluations stay authoritative
Model suggestions are Zod-validated, then checked against the deterministic safety policy.
Deployment is a hard gate
The Agent Pack cannot be generated while even one regression remains.
Inspectable artifacts
Download policy, skill, API contract, evals, test summary, README, and changelog as one ZIP.
Show it once. Correct it once.
Then prove it before you deploy it everywhere.