Stage 0: Contain the Coding Agent and Record a Delivery Baseline

This is the first playbook on the AgenticAdoptionLearningPath. It takes about an afternoon. It has two halves: limit what the agent can touch, and write down how work gets delivered today.

Do both before the agent does any real work. Containment is what makes early mistakes cheap. The baseline is what makes later claims checkable.

Each step below gives the control, the reason for it, and a way to check it. The reasons matter more than the steps: tools differ, and if you understand what a control is for you can find its equivalent in yours.

Why an agent needs limits that a careful colleague would not

It is fair to ask why any of this is needed. A new colleague is not denied read access to the environment file. Three things are different about an agent.

None of these happens often. But a session contains hundreds of actions and you will run many sessions, so "rare per action" still means "eventually". Since some of the outcomes cannot be undone, the sensible response is to make them impossible, which is cheaper than being careful forever.

Why an instruction to the agent is not a control

The obvious alternative is to write "never read secrets, never push to main" in the prompt or instruction file. Do write it, since it lowers how often the limits are tested. It cannot be the limit itself, for three reasons.

So every item below is enforced somewhere the agent cannot argue with: the tool's permission system, the operating system, or the hosting platform. And every item has a check, because a control you have not seen refuse something is a control you are assuming.

Containment checklist: what the agent can read, run and push

1. Secrets are out of reach

Why would an agent read a secrets file at all? Because reading configuration is what a competent debugger does. A connection fails, so it looks at how the connection is configured. Once read, the contents are in the conversation. From there they are sent to the model provider, and they can be repeated into places you did not intend: a log line, a test fixture, a commit message, a summary. Nothing has to go wrong for this to happen; it is the ordinary path.

Why no production credentials in the shell, if the agent is only working on development tasks? Because a command does not know which environment you meant. "Run the migration" runs against whatever the environment points at. If the shell can reach production, a routine debugging step can.

2. The main branch cannot be changed directly

Why, if every diff gets read anyway? Two reasons. First, reading every diff is a habit, and habits lapse on busy days; protection holds on those days too. Second, agents run git commands as part of ordinary work, and when a branch has diverged, a force push is one way to make the error go away. With protection on, the worst case is a bad branch, which is deleted in a second.

Why does this belong in stage 0 and not with the tests in stage 1? Because the tests you build next only have force if something makes them mandatory. Required checks on a protected branch are that something. Without it, a failing test is advice.

3. Commands that cannot be undone need a person

Why treat these differently from other commands? The costs are lopsided. Approving a command takes a few seconds. A wrong delete or a rewritten history can cost hours, or be unrecoverable. Where being wrong is that much more expensive than being slow, a pause is cheap insurance.

Then why not ask for everything? Because prompts only work if they are read. Asked fifty times an hour to approve git status and test runs, anyone starts approving without looking, and the one prompt that mattered goes through with the rest. Allow-listing the harmless commands is what keeps the remaining prompts rare enough to get real attention.

4. The agent works in a disposable place

Why does disposability matter? Partly to protect your own uncommitted work from an agent editing the same files. Mostly because of what it does to your judgement. When discarding a session costs nothing, you discard bad ones. When it would also lose your own changes, you try to rescue them, and rescuing a session that went wrong early usually takes longer than starting again with a better instruction.

5. Outbound reach is known

Why trim network access? Leaking something takes two abilities: reading it and sending it somewhere. An agent that can do both, and that reads text written by a stranger, has everything needed for that stranger to extract data. Removing either ability breaks the chain. Step 1 limits reading; this step limits sending.

Why raise the data question now? Because the answer can end the whole effort, and it is far better to learn that in the first week than after months of work that has to be unwound. It is also not the agent's risk but yours: the code leaves your machine in every session by design.

6. Spending has a ceiling

Why? An agent stuck retrying the same failing approach does not get bored or notice the time. It will continue until something stops it. Cost is also one of the things you will be asked about later, and a surprise bill in the first month is a poor start to that conversation.

Why the first month is attended and every diff is read

This is slow, and the slowness has a purpose. You do not yet know where the agent is reliable in your codebase and where it is not. That varies by codebase far more than by tool: an agent that is dependable in your API layer may be consistently wrong about your build scripts or your oldest module. Reading every diff is how you draw that map. Once you have it, you can relax attention where the agent has earned it and keep it where it has not. Relaxing before you have the map means relaxing everywhere at once, including the places that will hurt.

The third rule exists because an exception for agent work, once made, tends to become the normal route.

The baseline: four delivery numbers and one cost

Record these for the slice of code you will work in, or for the whole codebase if the slice is too small to measure. Use the last 8 to 12 weeks. Rough numbers taken from your existing tools are enough.

MeasureWhat to recordWhere it comes from
Lead timeMedian time from first commit on a change to mergePull request history
CI healthMedian duration of the required checks, and how often a run fails then passes on retry with no code changeCI history
Escaped defectsNumber of bugs found after merge, and how many were traced to a change in this periodIssue tracker
Review loadPull requests reviewed per person per week, and median time to first reviewPull request history
CostCurrent spend on AI tooling, if anyBilling

Add two notes in words:

Why these four measures and not others

Each one corresponds to a specific way adoption goes right or wrong. Together they make it hard for a gain in one place to hide a loss in another.

Lines of code, number of pull requests and "percentage of code written by AI" are left out on purpose. They measure activity, they go up whether or not anything improved, and an agent can inflate all three without effort.

Why the last 8 to 12 weeks, medians, and rough numbers

Why the baseline has to be written down before any agent work

The numbers themselves will still be in your tools later. Two things will not.

The first is the context. In three months nobody will remember that the baseline period included a migration, and the comparison will be misread.

The second is your impartiality. If the measures are chosen after the results are in, the choice drifts toward the ones that look good. That is not dishonesty; it is how people reason about things they have invested in. Fixing the measures in advance is the only reliable defence.

Why the baseline must never be used to compare people

The baseline describes a process. It is the "before" for every trial in AgenticTechniqueTrials.

If people believe the numbers will be used to judge them individually, they will manage the numbers: changes get split to shorten lead time, bugs get fixed quietly instead of reported. Each of those is a reasonable response to being measured, and each makes the numbers describe the measuring instead of the work. You then have no baseline. Say plainly, when you record it, what it will and will not be used for.

Expect the numbers to be noisy. A small group produces few data points a week, so small differences mean nothing. You are looking for changes large enough to see without statistics.

Exit test for stage 0

You are done when all of these are true:

  1. The agent's attempt to read a secrets file is blocked by the tool.
  2. The agent's attempt to push to the main branch is rejected by the platform.
  3. You can discard the agent's working copy without loss.
  4. A spending limit or alert is set.
  5. The four delivery numbers, the cost, and the two notes are written somewhere they will be found later.

Items 1 and 2 are tested by attempting the forbidden thing, for the reason given earlier: a control is only known to work once it has been seen to refuse.

Then move to AgentBuiltTestHarness.

See Also