Contents
- Why a runbook turns chaos into protocol
- Structuring the runbook by incident type
- The detection and triage section
- The blacklist listing procedure
- The Gmail spam classification procedure
- The ISP block procedure
- The data sources every step depends on
- Roles and keeping the runbook current
- Exercising the runbook
Why a runbook turns chaos into protocol
A deliverability incident does not wait for a convenient moment. The blacklist listing, the Gmail spam classification, the ISP block, these tend to surface when mail stops getting through, the business notices, and someone has to respond, often outside of calm working hours. In that moment, the difference between a fast, correct response and a slow, error-prone one is whether the responder is following a plan or improvising.
This operational note is about building that plan, a deliverability runbook. The structure of this note: why a runbook turns chaos into protocol, structuring the runbook by incident type, the detection and triage section, the blacklist listing procedure, the Gmail spam classification procedure, the ISP block procedure, the data sources every step depends on, roles and keeping the runbook current, and how the runbook is exercised so it works when it is needed.
The core argument: a deliverability runbook is a documented set of response procedures defining exactly what an operator does when an incident occurs. Without one, the responder improvises under stress, trying to recall the right sequence while mail is down, which is slow and inconsistent. With one, the responder executes a procedure written calmly in advance, by someone who had time to think it through. The runbook turns the incident from an improvisation into the execution of a known plan, and that materially reduces both the response time and the damage.
Structuring the runbook by incident type
A deliverability runbook should be structured by incident type, with a separate procedure for each kind of incident, because different incidents require genuinely different responses.
The main incident types a deliverability runbook should cover:
| Incident type | What has happened |
|---|---|
| Blacklist listing | A sending IP or domain is on a blocklist |
| Gmail spam classification | Gmail is filtering the operation's mail to spam |
| ISP block / throttling | A receiver is rejecting or rate-limiting the mail |
| Reputation decline | Sending reputation is degrading across receivers |
Each gets its own procedure. A single generic procedure cannot serve them all well, because a blacklist listing, a Gmail reputation problem, and an ISP block have different correct responses.
Within each incident-type procedure, the structure should be a clear, ordered sequence: how to confirm and characterize the incident, what data sources to check and what to look for, what immediate actions contain the damage, how to identify and fix the root cause, and how to recover and confirm recovery. The procedure should also say who is responsible for each part.
Beyond the per-incident procedures, the runbook needs a general section, covered next, on how an incident is detected and triaged in the first place. Structured this way, by incident type with ordered steps, the runbook is usable under pressure: a responder facing a blacklist listing turns to the blacklist procedure and follows it.
The detection and triage section
Before any incident-type procedure applies, the incident has to be noticed and identified. The runbook's detection and triage section covers this.
How an incident is detected. The runbook should state how deliverability incidents are detected: the monitoring that watches for them, blacklist monitoring, the accounting log's delivery and deferral rates, Gmail Postmaster Tools, Microsoft SNDS, and the alerts those generate. A responder should know that an incident has been detected because a specific monitor fired, not because someone happened to notice.
How an incident is triaged. When something is detected, the triage step identifies which kind of incident it is, which determines which procedure applies. The triage asks: is this a blacklist listing, a Gmail reputation problem, an ISP block, a broader reputation decline? The runbook should give the responder a quick way to characterize the incident and route to the right procedure.
How an incident is escalated and communicated. The section should cover when and how an incident is escalated, who is brought in for a serious incident, and how it is communicated, to the team, to the business, to any affected customers. A deliverability incident affecting mail delivery has business impact, and the communication is part of the response.
The detection and triage section is the runbook's front door: it gets the responder from "something is wrong" to "this is a blacklist incident, go to that procedure" quickly and reliably.
The blacklist listing procedure
The blacklist procedure handles a sending IP or domain being added to a blocklist.
The ordered steps:
- Confirm and identify the listing. Confirm the listing and identify which blocklist, and for Spamhaus which sub-list, using the Spamhaus reputation checker and a multi-blocklist check. Read the stated reason for the listing.
- Triage by impact. Determine whether the listing actually matters. A Spamhaus or Barracuda listing is urgent; some minor blocklists can be lower priority.
- Take the listed IP out of rotation. Stop sending from the listed IP, pausing its queues, so the listed IP is not used while the incident is handled. A pool means the other IPs continue.
- Identify and fix the root cause. Determine what caused the listing, spam, a compromise, a list-quality problem, a volume spike, and fix it thoroughly. This is the substance of the response.
- Request delisting only after the fix. Once the cause is confirmed fixed, request delisting through the blocklist's process. Requesting delisting before the fix leads to re-listing.
- Confirm and recover. Confirm the delisting took effect, and return the IP to sending gradually.
The procedure embeds the key principle, fix the root cause before requesting delisting, so a responder following it does the steps in the order that actually works.
The Gmail spam classification procedure
The Gmail procedure handles Gmail filtering the operation's mail to spam, a Gmail reputation problem.
The ordered steps:
- Check the spam rate in Postmaster Tools. The user-reported spam rate is the first thing to check. A rate above the 0.10 percent working ceiling, and certainly approaching 0.30 percent, is the likely cause.
- Check the Compliance Status. Postmaster Tools v2's Compliance Status shows whether the bulk sender requirements are met. A non-compliant indicator points at the specific gap.
- Check authentication. Confirm SPF, DKIM, and DMARC with alignment are all passing for the Gmail mail.
- Read the accounting log for Gmail. Check the deferrals and rejections from gmail.com for corroboration.
- Identify the cause. A high spam rate points at list quality and engagement; an authentication failure points at the SPF, DKIM, or DMARC setup; a non-compliant status points at the unmet requirement.
- Address the cause and reduce volume. Fix the cause, and reduce Gmail volume while recovering so the operation is not pushing into the degraded state.
- Monitor the recovery. Watch the spam rate and Compliance Status in Postmaster Tools as the recovery progresses; a spam-rate recovery is slow.
The procedure routes the responder to the spam rate first, because the spam rate is the most common Gmail deliverability cause and the metric only Postmaster Tools shows.
The ISP block procedure
The ISP block procedure handles a specific receiver, Microsoft, Yahoo, or another, blocking or throttling the operation's mail.
The ordered steps:
- Identify the receiver and the nature of the block. Determine which receiver is involved and whether the mail is being rejected outright or throttled, from the accounting log's responses for that receiver.
- Read the rejection or deferral text. The receiver's SMTP response and diagnostic text frequently state the reason. A Microsoft 550 5.7.515 points at a bulk-sender-requirement failure; 421 deferrals point at throttling.
- Check the receiver's reputation tools. For Microsoft, check SNDS. For other receivers, check their available tools. A poor reputation reading explains a block.
- Check authentication and compliance. Confirm authentication is correct and the bulk sender requirements are met for that receiver.
- Identify and address the cause. A reputation problem points at list quality, complaints, and engagement; an authentication or compliance gap points at the specific unmet requirement.
- Reduce volume and use the support process. Reduce volume to the receiver while recovering, and, if the receiver provides a deliverability support process, use it once the cause is fixed.
- Monitor the recovery. Watch the receiver's reputation tools and the accounting log as the recovery progresses.
Because different receivers have different tools and specifics, the ISP block procedure can have sub-sections or notes for the major receivers, so a responder dealing with a Microsoft block is pointed at SNDS and the Microsoft specifics.
The data sources every step depends on
Every step in every procedure depends on a data source, and the runbook must specify, for each step, which data source to use and how to access it.
| Data source | Used for |
|---|---|
| The PowerMTA accounting log | Delivery, deferral, bounce, and rejection data per receiver |
| Gmail Postmaster Tools | The Gmail spam rate, Compliance Status, authentication |
| Microsoft SNDS | The Microsoft reputation reading and complaint data |
| Blacklist checkers | Whether IPs are listed and on which blocklists |
| The monitoring dashboards | The trended queue, delivery, and reputation metrics |
The runbook should not just name these sources but make them immediately usable in an incident: the actual URLs, the login or access details, the location of the dashboards, where the accounting log is and how to query it. A responder in an incident should not be hunting for how to access SNDS; the runbook should give them the link and the access path.
A common weakness in a deliverability runbook is steps that say what to check but not where. A step that says check the spam rate, without saying that means Gmail Postmaster Tools, here is the URL, here is who has access, forces the responder to work that out during the incident, which is exactly when there is no time. Every runbook step should name its data source explicitly and give the responder what they need to reach it. The runbook is a tool for speed under pressure, and a step that sends the responder off to figure out the data source themselves undermines that. Specify the data source, the access, and what to look for, for every step.
Roles and keeping the runbook current
Roles. The runbook should specify who does what in an incident. Even in a small team, the runbook should be clear on who is the responder for a deliverability incident, who is escalated to for a serious one, who handles the communication to the business, and who has access to the various data sources and the ability to take the actions, pausing queues, changing configuration. In an incident, ambiguity about who is doing what costs time. The runbook removing that ambiguity is part of its value.
Keeping the runbook current. A runbook is a living document, kept current by regular review and post-incident updates, not written once and left to age. A runbook goes stale because the things it references change: the receivers change their requirements and tools, Gmail moved to Postmaster Tools v2, Microsoft changed how SNDS and JMRP are administered, the thresholds shift; the operation's own infrastructure changes; and the team changes. A runbook referencing a retired tool, an old threshold, or a former team member is misleading exactly when it is needed.
The practices that keep a runbook current:
- A scheduled periodic review, a couple of times a year, reading the runbook through against current reality and updating where it has drifted.
- A post-incident update, every time the runbook is used in a real incident, the responders note what was unclear or missing or wrong, and the runbook is updated, so each incident improves it.
- Updates tied to change, any relevant infrastructure or team change triggers a corresponding runbook update.
A runbook maintained this way stays accurate and trustworthy. One written once and never revisited becomes, within a year or two, a document the team cannot rely on, which defeats its purpose.
Exercising the runbook
A runbook that has never been used until a real incident is an untested runbook, and an incident is a bad time to discover the runbook has gaps. Exercising the runbook, practicing it outside of a real incident, is what gives confidence that it works.
The ways to exercise a runbook:
Walk-throughs. The team walks through a procedure together, reading it step by step, discussing whether each step is clear, whether the data sources are right, whether anything is missing. This is low-effort and catches obvious gaps.
Tabletop exercises. The team runs a simulated incident: someone poses a scenario, "a sending IP has just been listed on Spamhaus," and the responders work through the runbook procedure as if it were real, checking that the procedure leads them correctly. This tests the procedure more realistically than a walk-through.
Using it for real, minor incidents. When a small, real deliverability issue occurs, using the runbook for it, even though the issue is minor, exercises the runbook and the team's familiarity with it.
Exercising the runbook does two things. It finds gaps, a missing step, an unclear instruction, a wrong data source, while there is time to fix them calmly, rather than during a real incident. And it builds the team's familiarity, so when a real incident happens, the responders are executing a procedure they have practiced, not reading an unfamiliar document under stress. A runbook that has been exercised is a runbook the team can actually rely on.
An operation we worked with had a deliverability incident: one of their sending IPs was listed on Spamhaus, and their delivery to a range of receivers dropped sharply as a result. They did not have a deliverability runbook. The person who happened to be available to respond was capable but was improvising. They spent time working out that the problem was a blacklist listing rather than something else, time working out which blocklist and how to check it, time deciding what to do, and in their improvised response they made the classic mistake the absence of a runbook invites: they found the Spamhaus self-service removal and used it immediately, before identifying and fixing the root cause. The IP was re-listed shortly after, because the underlying cause, a bad list segment generating spam-trap hits, was still active. They went around the delist-relist loop more than once before someone stepped back and did the root-cause work. The whole incident took far longer to resolve than it should have, and the repeated delisting attempts made it worse. Afterward, we helped them build a deliverability runbook, and the blacklist procedure in it would have changed the entire incident. The procedure would have told the responder, in order: confirm the listing and identify the blocklist with the Spamhaus checker, here is the link; take the listed IP out of rotation; identify and fix the root cause; and only then request delisting. A responder following that procedure would not have rushed to the delist form, because the procedure explicitly places the delisting request after the root-cause fix. The incident that took them a long, painful improvisation would have been a contained, ordered response of confirm, contain, fix, delist. The lesson is the core argument for a runbook. The knowledge of how to handle a blacklist listing correctly existed, it is well established, but in the moment of the incident, improvising, the responder did not apply it correctly. A runbook is how that established knowledge is made available, in the right order, at the moment it is needed, so the response is a protocol rather than an improvisation. Building the runbook before the incident is what turns the knowledge into a reliable response.
A deliverability runbook is a documented set of response procedures that turns a deliverability incident from a stressful improvisation into the execution of a known plan, and building one is among the higher-value operational preparations a sending operation can make. The runbook should be structured by incident type, a specific procedure for a blacklist listing, for a Gmail spam classification, for an ISP block, for a reputation decline, each with ordered steps covering confirmation, containment, root-cause fixing, and recovery, plus a general detection and triage section that routes a detected incident to the right procedure. Every step must name its data source explicitly, the accounting log, Postmaster Tools, SNDS, the blacklist checkers, with the access details, so the responder is not hunting for sources mid-incident. The runbook must specify roles, and it must be kept current through scheduled review and post-incident updates, because a runbook referencing retired tools or old thresholds misleads exactly when it is needed. And it should be exercised, through walk-throughs and tabletop scenarios, so its gaps are found calmly and the team is familiar with it. Operations that build, maintain, and exercise a deliverability runbook respond to incidents fast and correctly; operations without one, as the case shows, improvise under pressure and make the avoidable mistakes a runbook exists to prevent.