New to control systems, or need a refresher? Start with Control Systems: A Primer.
I run AI agents that operate my businesses day to day. They read the numbers every morning, draft campaigns and replies, and ask me to approve anything that touches a customer or money. The open question is how much of that asking can stop. I answer it the way I'd commission a new control loop on the plant floor: start in manual, measure, and raise the gain only as fast as the measurements justify.
This is a different problem from the one in My Framework for Working with AI Agents. There, the agent works inside a repository, everything it does is in version control, and full permissions are the right call. An operations agent acts on the outside world: it emails prospects and changes ad spend. Those actions can't be reverted with git, so its autonomy has to be earned.
The loop
Each action the agent proposes goes around the same loop:
| Control element | Agent equivalent |
|---|---|
| Setpoint | The outcome the agent predicts: "bounce rate on this campaign stays under 2% for 7 days" |
| Controller output | The action it proposes, such as pausing a campaign or launching an audience |
| Gate | My approve or reject, with an optional note |
| Plant | The business: campaigns, ad accounts, prospects |
| Sensor | The real metric, read when the prediction comes due |
| Error | The grade: hit, partial or miss |
Figure 1. The inner loop (white and blue) is the agent running the business. The outer loop (amber) grades its predictions and sets the gain K. When K = 0, an action only passes once I approve it; when K = 1, it goes straight through. The daily cap and the interlocks sit between the gate and the plant.
The rule that makes the loop work: every request is a prediction. The agent can't ask for anything without stating what will happen, by when, and how sure it is, as a probability . A vague request can't be graded, and a loop with no measurement is open loop.
What gets measured
Three numbers per action type tell me whether the agent is ready for more gain:
- Approval rate. How often I agree with what it asks for. This measures whether its judgment matches mine.
- Hit rate. How often its predictions came true, scoring a partial as half. This measures whether its model of the business is right.
- Calibration. Its Brier score, , where is 1 for a hit and 0 for a miss. Zero is perfect. Saying 0.5 on everything scores 0.25, a coin flip. An agent that says 90% and is right 60% of the time is a sensor reading high, and I want to know that before I trust it.
The agent grades its own predictions, because it has the data and the time. Self-reported grades are a sensor that needs a calibration check, though. So once a week I get the grades as a single approval: approve confirms them, and reject with a note disputes them. A disputed grade doesn't count until the agent regrades it.
Raising the gain
Autonomy is set per action type, not for the agent as a whole. Pausing a campaign and resuming one are different decisions with different risks, so they're scored separately. A type becomes eligible for autonomy when, over its recent history:
- I've decided on it at least 10 times, approving at least 90%.
- At least 5 outcomes are graded, with at least 80% hits.
- The last 5 are clean: no rejection, no miss, nothing disputed.
Eligibility only produces a recommendation. I raise the gain myself by adding the type to a file the agent can't edit. From then on it acts and tells me afterwards, and it is scored on exactly the same terms as before.
That's how you bring a new loop online. Run it in manual, then in auto with conservative tuning, then tighten as the trend data shows it holding. You don't jump to aggressive tuning because the first day went well.
Protection
Gain alone isn't safe. Three protections sit around it.
A circuit breaker. If any of a promoted type's last 3 graded outcomes is a miss or disputed, its autonomy switches off on its own and requests go back to me. Like a tripped breaker, it doesn't reset itself: I look at what happened first. A loop that just made a bad call is the last one that should keep acting unattended.
A rate limit. Each promoted type has a daily cap. It works like actuator saturation: even a correct controller shouldn't be able to slam the valve.
Hardwired interlocks. Some actions are never promotable, whatever the score. These are spending money, first contact with a customer, and anything that can't be undone. On a plant, safety functions don't live in the same program as process control, and operators can't edit them from the HMI. The same rule applies here. The never-promote list, the autonomy file and the agent's own toolbelt all sit where the agent can't write, and the runner denies edits to its own code. An agent that could edit its interlocks wouldn't have any.
Why bother
Without this, the two options are bad. You approve everything forever, and the agent saves you very little. Or you hand it everything at once on a feeling, and you learn its failure modes from customers. The loop gives a third option: autonomy that grows with the evidence, shrinks on its own when the evidence turns, and never reaches the things that should always have a person on them.
It's the same reasoning I use for money in Money as a Control System: measure the gain before you raise it, respect the delay, and keep the hard limits outside the controller.