How to Run a Deliverability Audit Without a Third-Party Tool

• May 2023
• Engineering Memo · External Release

The email deliverability tool market offers dozens of platforms that promise full audits, inbox placement testing, and reputation monitoring. Many are useful. None are necessary for an operator with access to the data sources already present in a production PowerMTA environment. The data required for a thorough deliverability audit — SMTP-level delivery statistics, ISP reputation signals, authentication status, complaint rates, and bounce analysis — exists in the accounting log, Postmaster Tools, SNDS, DMARC aggregate reports, and FBL data. The audit methodology described here uses only these native sources.

This is not an argument against third-party tools — seed testing and dedicated deliverability platforms provide value that native data sources cannot, particularly for per-campaign inbox placement measurement. It is an argument that most deliverability questions can be answered without them, and that operators who understand native data sources are better equipped to interpret third-party tool output when they do use it.

The Six-Section Audit Structure

A complete deliverability audit covers six sections: authentication status, infrastructure reputation, delivery performance, complaint rate, bounce rate, and list quality indicators. Each section draws on specific data sources and answers specific questions. The audit is designed to be completed in 2–4 hours with direct access to all data sources, producing a written summary that documents current status and identifies specific action items.

Section 1: Authentication Status

Data sources: DMARC aggregate reports (last 30 days), DNS TXT record lookups, test message Authentication-Results header.

What to check: DMARC pass rate across all sources. Any source IP appearing in DMARC reports that was not expected. DKIM alignment pass rate separately from SPF alignment pass rate. SPF lookup count (should be ≤10). DMARC policy level (p=none, p=quarantine, p=reject). DKIM key size for each active selector (should be 2048-bit for bulk senders).

Rating criteria: Good = DMARC pass rate ≥99%, all sources expected, DKIM 2048-bit, DMARC at p=quarantine or p=reject. Caution = DMARC pass rate 97–99%, minor SPF coverage gaps, p=none but in transition. Problem = DMARC pass rate below 97%, authentication failures from unknown sources, 1024-bit DKIM keys, no DMARC record.

Section 2: Infrastructure Reputation

Data sources: Gmail Postmaster Tools domain reputation and IP reputation sections; Microsoft SNDS for each sending IP; DNSBL check for each sending IP against Spamhaus (SBL, XBL, PBL), Barracuda, SORBS, SpamCop.

What to check: Gmail domain reputation tier (High/Medium/Low/Bad). Gmail IP reputation tier for each registered IP. SNDS status for each IP (Green/Yellow/Red). DNSBL status — any listing on any major list requires immediate attention regardless of other signals. Gmail spam rate trend over the past 30 days (direction and absolute value).

Rating criteria: Good = High domain reputation, no DNSBL listings, SNDS Green. Caution = Medium domain reputation, one SNDS Yellow, no DNSBL listings. Problem = Low or Bad domain reputation, any DNSBL listing, SNDS Red on any IP.

Section 3: Delivery Performance

Data sources: PowerMTA accounting log — last 90 days of delivery data aggregated by ISP destination and virtual MTA.

What to check: Overall delivery rate (250 OK / total attempts). Per-ISP delivery rates for the top 5 ISPs by volume. Per-ISP deferral rates and trends. SMTP attempts per delivered message (ratio indicating retry pressure). Queue depth patterns during campaign windows. Any ISP with delivery rate significantly below programme average.

Sections 4–6: Complaint Rate, Bounce Rate, List Quality

Section 4 — Complaint Rate. Check Gmail Postmaster Tools spam rate for the past 30 days. Check Yahoo FBL complaint count per campaign (from the FBL processing log). Calculate complaint rate as: complaints / delivered messages. Rating: Good <0.05%; Caution 0.05–0.08%; Problem >0.08%. Note any campaigns with complaint rates above programme average — these are the source of the issue and require list segment investigation.

Section 5 — Bounce Rate. From the accounting log: count 5XX responses per campaign over the past 30 days. Calculate hard bounce rate: 5XX responses / total injection attempts. Rating: Good <0.3%; Caution 0.3–0.5%; Problem >0.5%. Segment by ISP to identify whether the bounce problem is programme-wide or ISP-specific. Segment by list source to identify whether the problem is concentrated in specific acquisition sources.

Section 6 — List Quality. Calculate the percentage of the active list that has not opened or clicked in the past 90 days. Calculate average list age (time since acquisition) for the active list. Identify the top 3 acquisition sources by volume and their respective hard bounce rates. Rating: Good = less than 30% inactive for 90 days, bounce rates consistent across acquisition sources. Caution = 30–50% inactive, minor bounce rate variation across sources. Problem = more than 50% inactive, bounce rate above 1% from any acquisition source.

Producing the Audit Report

The audit output should be a written document — not a spreadsheet or dashboard — that a business stakeholder can read and understand without technical background. The structure: one paragraph per section summarising the findings, a rating (Good/Caution/Problem) for each section, and a prioritised list of action items from most to least urgent.

Prioritisation logic: any Problem rating generates an immediate action item, regardless of which section it appears in. A DNSBL listing and a Gmail domain reputation of Low are both P1 regardless of whether all other sections are Good. Any Caution rating generates a 30-day action item — it must be investigated and resolved before the next quarterly audit. Good ratings for all sections with no Caution results produce a maintenance action item to schedule the next quarterly audit.

The audit should be repeated quarterly — or after any significant programme change (new list acquisition, new campaign type, infrastructure migration) that could affect any of the six sections. A programme that conducts quarterly audits using this framework and acts on the findings consistently will not experience unexpected inbox placement deterioration — because the audit detects deterioration at the Caution stage, before it progresses to Problem.

Running the Authentication Section: Step by Step

The authentication section is the fastest to run and the most important to get right, because authentication failures affect every message regardless of other factors. Begin by downloading the DMARC aggregate reports for the past 30 days from the RUA mailbox. If your programme uses a DMARC reporting service (dmarcian, Valimail, Google Postmaster DMARC Insights), the data is already parsed and filterable. If processing raw XML, an open-source parser (parsedmarc, dmarc-report-viewer) converts the XML to CSV for analysis.

From the aggregate report data, extract: the distinct source IP addresses that sent mail from your domain; the DKIM alignment pass rate per source; the SPF alignment pass rate per source; and any sources with alignment pass rates below 99%. Sources with consistent 100% pass rates require no immediate action. Sources with any failures require investigation: is this source expected to be sending from your domain? If yes, why is authentication failing? If no, it may be a spoofing source or an unauthorised sender that should be blocked by DMARC policy.

Next, check the SPF lookup count. Using mxtoolbox.com/spf.aspx or a similar tool, enter your sending domain and examine the lookup count. If it exceeds 10, the SPF record will fail at some or all ISPs that strictly enforce the limit. Identify which includes are contributing the most lookups and plan the flattening or removal of the highest-cost includes. Check that the IP address range of your primary sending infrastructure is covered by the SPF record — not just the include: mechanisms, but the actual IP addresses that outgoing mail uses.

Check DKIM by sending a test message from each configured sending source and inspecting the Authentication-Results header in the received message. The header format will show: dkim=pass (or dkim=fail), the selector used (s=mail2024 or similar), and the domain (d=yourdomain.com). A dkim=fail response indicates the signing is misconfigured or the DNS record does not match the active signing key. Resolve before moving to other audit sections — authentication failures cascade into other problems and invalidate the findings of other sections.

Running the Infrastructure Reputation Section

Open Gmail Postmaster Tools and select your primary sending domain. Check the domain reputation classification on the dashboard. Export the spam rate graph for the past 30 days — look for trend direction (improving, stable, or declining) and any spikes that correspond to specific campaign dates. Check the IP reputation section for each registered sending IP.

DNSBL checking: query each sending IP against Spamhaus DBL (domain-based), SBL (spam source), XBL (exploits), and ZEN (combined check); Barracuda; and SpamCop. The most critical check is Spamhaus ZEN — a listing on Spamhaus's SBL or XBL is the most impactful DNSBL listing for email delivery, as virtually all major ISPs query Spamhaus in their inbound filtering. A Spamhaus listing on any active sending IP should be treated as a P1 incident.

The SNDS check for Microsoft: log into the SNDS portal (postmaster.live.com) and check the status colour for each sending IP. Green means acceptable reputation with Microsoft. Yellow means the IP has elevated complaint rates or other concern signals — investigate the per-IP complaint data available in SNDS. Red means the IP is currently blocked at Microsoft — investigate the block reason and initiate the unblock request process through SNDS before sending any further messages through the affected IP.

Running the Delivery Performance Section

The accounting log analysis requires a query language capable of aggregating across the log fields: sending IP, destination domain, SMTP disposition (delivered, bounced, deferred), and SMTP response code. PowerMTA's accounting log is in CSV format, processable with standard tools: Python (pandas), SQL (if the log is imported to a database), or even Excel pivot tables for smaller log volumes.

The queries you need: (1) delivered messages / total attempts per ISP for the past 30 days; (2) deferral count per ISP for the past 30 days, showing trend by week; (3) average SMTP attempts per delivered message per ISP (divide total SMTP attempts by delivered messages — this ratio catches retry pressure that aggregate delivery rate hides); (4) hard bounce count per ISP and per list segment for the past 30 days; (5) any ISP that returned authentication-related rejection codes (5.7.x responses referencing authentication or policy failures) during the period.

These five queries, with their results compared against the thresholds in Table 1, produce the Section 3 rating. Document any ISP with a Caution or Problem rating, along with the specific metric values that produced the rating and the date range of the data. This documentation provides the baseline for the next audit's comparison — without it, the next audit cannot determine whether the metric improved, remained stable, or worsened.

Making the Audit Actionable

The value of a deliverability audit is not in the status ratings themselves — it is in the specific actions those ratings produce. A Problem rating on authentication without a specific action item ("update SPF record to add the new CRM platform's include before next campaign") is a description of a problem, not a plan for resolving it. Every Problem and Caution rating must produce a specific, assigned, time-bound action item.

The action item format: [Specific action] + [Owner] + [Due date] + [Success criteria]. Example: "Reduce SPF lookup count from 13 to ≤9 by removing the deprecated Pardot include: and flattening the Salesforce include: to IP list. Owner: infrastructure team. Due: within 7 days. Success criteria: SPF lookup count ≤9 verified by mxtoolbox.com/spf.aspx." This format ensures the action is concrete, assigned, and measurable — not a vague "look into the SPF record issue."

Action items from the audit should be tracked in a shared system visible to both the infrastructure team and the email marketing team — because some action items (list quality improvements, campaign frequency reduction, new acquisition source validation) require marketing team actions, not infrastructure changes. The audit produces work for both functions, and tracking action items separately in function-specific systems produces the situation where infrastructure fixes its items while marketing's items remain open — resulting in continued deliverability problems from the unaddressed marketing-side factors.

A quarterly audit cadence, with the prior quarter's action items reviewed at the start of each new audit, creates a continuous improvement loop. Problems identified in Q1 are resolved by Q2; Q2's audit verifies the resolution and identifies any new issues; Q3 audits against the Q2 baseline. Over four quarters, this process typically produces measurable improvement in every section — authentication pass rates approaching 100%, domain reputation stable at High, delivery performance metrics trending toward the Good thresholds, complaint rates below the concern threshold, and bounce rates demonstrating consistent list hygiene progress.

The native data sources — PowerMTA accounting logs, Postmaster Tools, SNDS, DMARC reports, FBL data — provide all the information needed for this process. They are available in every production PowerMTA environment, they are updated continuously, and they reflect the actual state of the sending infrastructure's relationship with the ISPs that determine inbox placement. Operators who learn to read these sources fluently are equipped to understand and improve their deliverability without dependence on any particular third-party tool, while also being better positioned to interpret third-party tool output when they choose to use it.

Interpreting the Audit in Context

A deliverability audit produces a point-in-time snapshot. Its value increases when it is compared against previous audit data — the trend over time tells a more informative story than any single audit result. A programme whose hard bounce rate has declined from 0.8% to 0.4% over four quarters, with no other changes to rating, is on a positive trajectory. A programme whose complaint rate has increased from 0.04% to 0.07% over the same period, while maintaining all other Good ratings, has a developing problem that will become a Problem rating in the next audit without intervention.

The trend analysis requires that audit data be stored systematically across audit cycles — not just the action items, but the specific metric values that produced each rating. A simple tracking format: a spreadsheet with one row per audit per section, capturing the rating and the specific metric values. This data becomes increasingly useful after four or more audit cycles, when trend lines can be drawn and projections made about when a trending metric will cross a rating threshold.

Audit results should be shared with business stakeholders at a level of abstraction appropriate to their role. The infrastructure team receives the full technical report with specific metric values and action items. Email marketing leadership receives the section ratings and the action items that affect their campaign decisions. Executive stakeholders receive a summary sentence per section: "Authentication: Good. Domain reputation: Caution — trending toward Good with improved list hygiene. Hard bounce rate: Caution — requires action within 30 days." This tiered communication ensures the right information reaches the right decision-makers without overwhelming non-technical stakeholders with infrastructure-level detail.

The relationship between audit quality and programme outcomes is direct: programmes that conduct rigorous quarterly audits and implement all action items consistently outperform those that audit less frequently or less thoroughly. This is not correlation — it is causation. The audit identifies the specific factors limiting deliverability performance, the action items remedy those factors, and the next audit confirms the remediation before new issues accumulate. The programme never drifts far from optimal state because the audit cycle catches drift early and the action items reverse it before it compounds. This is the fundamental value of the systematic approach: not insight alone, but the disciplined response to insight that produces continuous improvement.

When Third-Party Tools Add Value the Native Audit Cannot

Having documented how to conduct a complete audit without third-party tools, it is appropriate to acknowledge where those tools provide genuine value that native data sources cannot replicate. The primary use case is per-campaign inbox placement measurement: knowing, for a specific campaign, what percentage of messages reached the inbox versus the spam folder at Gmail, Yahoo, and other major ISPs. Native data sources cannot answer this question — SMTP delivery rate confirms ISP acceptance, Postmaster Tools confirms domain reputation trend, but neither can tell you where a specific campaign's messages were placed after acceptance.

Seed testing services (GlockApps, Mail-Tester, Litmus Email Analytics) place test seed addresses in the send list and report on where those seed addresses received the campaign. This provides campaign-level inbox placement data that complements the native audit framework. The limitation — seed addresses may have different reputation profiles from the actual list — is managed by using fresh, non-overused seed addresses and interpreting seed results as directional indicators rather than precise measurements.

For programmes running 10+ campaigns per month or with inbox placement as a primary business KPI, the investment in a seed testing service is justified by the campaign-level granularity it provides. For programmes running 1–4 campaigns per month with stable, High-reputation sending domains, the native audit framework provides sufficient visibility and seed testing adds marginal value relative to its cost. The decision is programme-specific, but the native framework should always be the foundation — third-party tools are useful complements, not substitutes for understanding the data that is already available in the sending environment.

Reputation monitoring services (Sender Score, Validity, Return Path) provide aggregated ISP complaint and reputation data across multiple ISPs from a single interface. This is convenient but not uniquely informative for operators who already review Postmaster Tools, SNDS, Yahoo FBL, and DNSBL status directly. The native sources are more specific — they show exactly which ISP, exactly which IP, and exactly which metrics — while aggregated reputation scores are useful for a quick health check but can obscure the ISP-specific detail needed for action. Use aggregated reputation scores as a supplementary dashboard indicator, not as the primary reputation assessment tool.

The deliverability audit framework described here is not a one-time diagnostic — it is a repeating operational practice. Programmes that run quarterly audits using this six-section structure, document their findings systematically, implement all action items before the next audit, and compare trend data across audit cycles build the institutional knowledge and operational discipline that produce consistently high deliverability outcomes. The native data sources available in every PowerMTA environment are sufficient to maintain this discipline without external dependencies. When third-party tools are also used, the native framework provides the context that makes their output interpretable and actionable.