Runbook Template for an Automation

A runbook written by the person who built the thing is almost always a description of the system. What somebody woken at 2am needs is different: a short list of decisions they are allowed to make on their own, and explicit permission to make them.

So this template puts the stop procedure first and the architecture last, which is the reverse of the order it wants to be written in.

Why this exists

The failure mode is a 3am phone call to the builder. That is not a documentation problem. It is a single point of failure wearing a documentation costume, and it happens because the builder writes down what they find interesting — the design — and leaves out what they know without thinking, which is the part everybody else needs.

The single most-needed fact in an automation runbook is whether the job can be safely run again. “Can I just re-run it?” is the first question anyone asks at 2am, and it is the question almost no runbook answers. If the honest answer is “no, it will double-post”, that is not a gap in the document. It is a design defect you have just found for free, months before it costs you anything. Write it down anyway, with the manual cleanup steps underneath it, and put the fix on the backlog with a ticket number in the runbook.

The second thing missing is a boundary. The person on call does not need the full picture; they need to know what they are allowed to do. “You may stop it. You may re-run today’s batch. You may skip one failed record and flag it. You may not touch the mapping file, and you may not re-run yesterday.” Without that list, a careful person does nothing and waits until morning, and a confident person does something irreversible. Both outcomes are yours, not theirs.

How to use it

• Order the sections for the emergency, not for the reader’s understanding. Stop, blast radius, is it safe to re-run, what you may decide, three known failures, escalation. How it works is section seven and can wait until morning.
• Write each symptom the way the person experiences it, not the way the log reports it. “The Monday report is empty” beats “extract returned zero rows” every time, because the person reading is matching against what they were told on the phone.
• Put the literal strings in. Queue names, file paths, the exact error text to search for, the dashboard link. A runbook that says “check the logs” has not helped anyone.
• Give escalation a clock. “If it is not resolved in thirty minutes, call Kenny.” Without a number, people wait far too long out of politeness, and the politeness costs more than the call.
• Test it once, at handover. Break something in a test environment — revoke a credential is the easy one — hand the page to somebody who did not build it, and watch without speaking. Every question they have to ask you is a section you left out. Write each one down and fix it that day.

Where it goes wrong

Runbooks rot, and a rotted runbook is worse than none because it still gets trusted. Date it, name one owner, and record which version of the automation it was written against. An undated runbook is a trap.

It also cannot carry authority it was never given. If the person on call is not actually permitted to stop the process, reverse a payment or hold a customer email, writing it in the runbook does not make it so. Get the authority agreed by name, in advance, with the person who owns the consequence — otherwise the document is sending somebody into a conversation they are not allowed to have.

Do not write one for an automation nobody is on call for. If there is no rotation and no alert, a runbook is theater; fix the alerting first and the runbook second. And watch the length. Past roughly two pages nobody reads it at 2am, and if it genuinely needs more than that, the honest finding is that the system is too complicated to be operated by one document.