Diagnosing Authentication Drift in Long-Running Infrastructures

• November 2022
• Engineering Memo · External Release

Authentication drift is the progressive degradation of SPF, DKIM, and DMARC alignment that occurs in long-running email infrastructure without active monitoring and maintenance. It is one of the most common root causes of deliverability problems that appear suddenly — "nothing changed" — in programmes that have been operating stably for months or years. In fact, something did change: the infrastructure changed, the services integrated with it changed, or the accumulation of minor configuration decisions finally crossed a threshold that began affecting delivery.

This note documents the mechanisms of authentication drift, the specific failure patterns it produces, and the systematic diagnostic approach that identifies which specific records or services are causing failures before they cascade into delivery incidents.

How Authentication Drift Accumulates

Authentication drift is not a single event — it is the accumulation of small changes that individually appear harmless but collectively degrade authentication coverage. The most common mechanisms:

SPF record growth. The SPF TXT record for a domain has a 10-lookup limit (RFC 7208). Each mechanism that requires a DNS lookup — include:, a:, mx:, ptr: — counts against this limit. A domain that starts with a clean 3-lookup SPF record and adds Salesforce (3 lookups), a marketing automation platform (2 lookups), a customer support email system (2 lookups), and a billing tool (2 lookups) has quietly exceeded the 10-lookup limit without any single addition appearing problematic in isolation. ISPs that strictly enforce the lookup limit begin seeing SPF failures for messages from any source, while the domain owner believes SPF is correctly configured.

DKIM key abandonment. When email infrastructure migrates — to a new server, a new ESP, a new MTA version — the DKIM signing configuration often changes. Old selectors are published in DNS but are no longer actively used for signing. If the DNS records are not cleaned up, they accumulate stale public keys that no longer correspond to any active private key. More critically: if a migration moves to a new server without migrating the private key, the new server signs with a new key but the old DNS record still exists. Depending on the transition handling, some message streams may reference the old selector while the new server uses the new selector — producing DKIM failures on those streams.

Third-party service changes. Services that send email on behalf of a domain — CRM systems, marketing automation, customer support platforms — occasionally change their sending infrastructure. When they migrate their outgoing mail servers, their sending IPs change. If those IPs were explicitly listed in the SPF record (using ip4: mechanisms rather than include: mechanisms), the old IPs remain in the record (contributing to lookup count) while the new IPs are not yet authorised (producing SPF failures on messages from the new infrastructure). The service provider may not communicate this change proactively, and the domain owner discovers it when SPF failures start appearing in DMARC aggregate reports.

The Systematic Diagnostic Approach

When authentication failures appear in DMARC aggregate reports or in Postmaster Tools authentication data, the diagnosis requires working through each authentication mechanism separately before attempting any fixes. Attempting to fix a problem before correctly identifying which mechanism is failing — and in which message stream — produces incomplete remediation that leaves some failures unresolved.

Step 1: Establish the failure scope. DMARC aggregate reports show which sending sources (identified by source IP address) are producing DKIM and SPF alignment failures. A failure affecting all sources simultaneously points to a domain-level configuration problem (SPF record over lookup limit, DMARC policy misconfiguration). A failure affecting a specific IP range points to a specific sending service that changed its configuration or infrastructure. Narrowing the scope before investigating specific records prevents wasted diagnostic effort on mechanisms that are functioning correctly.

Step 2: Verify SPF lookup depth. Use an SPF checker that counts actual DNS lookup depth — not just syntax validation. The SPF record at the sending domain must resolve within 10 lookups including all nested includes. If the lookup count exceeds 10, identify which includes are contributing the most lookups. Services that use include: mechanisms pointing to records with further nested includes are the top-priority targets for lookup reduction. SPF flattening — converting include: references to explicit ip4: lists — reduces lookup count but requires maintenance as the service's IPs change.

Step 3: Validate active DKIM selectors. For each DKIM selector currently published in DNS, verify that the corresponding private key is actively used for signing by the current sending infrastructure. Send a test message through each configured sending source and inspect the DKIM-Signature header to confirm: (a) a DKIM signature is present, (b) the selector referenced in the signature matches an existing DNS record, and (c) the signature verifies correctly. If any sending source produces no DKIM signature or references a selector that does not exist in DNS, that source has a signing configuration problem.

The DMARC Aggregate Report as a Drift Detection Tool

DMARC aggregate reports are the primary tool for detecting authentication drift before it becomes a delivery problem. These reports, sent daily by ISPs to the RUA address in the DMARC record, contain per-source-IP counts of DKIM alignment passes and failures, SPF alignment passes and failures, and DMARC policy evaluation outcomes. A programme that reviews DMARC reports regularly catches authentication drift at the first appearance — before it has accumulated enough volume to affect Postmaster Tools reputation or inbox placement metrics.

Parsing DMARC aggregate reports requires either dedicated software (dmarcian, Valimail, PowerDMARC, or open-source parsers) or manual XML parsing. The XML format is standardised (RFC 7489) but verbose — large programmes with many sending sources receive reports that contain thousands of rows per day. Automated parsing with alerting on newly appearing source IPs or on alignment pass rate drops below threshold is the correct approach for programmes with significant sending volume.

The key patterns to monitor in DMARC aggregate reports for authentication drift detection: (1) new source IP addresses appearing that were not in previous reports — these indicate a new or changed sending service that may not be properly authenticated; (2) a known source IP's alignment pass rate declining from 100% to below 99% — indicating an authentication change on that source's infrastructure; (3) a source IP appearing with consistent alignment failures for more than 3 days — indicating a configuration problem that is not self-resolving and requires investigation.

Preventing Drift Through Process Controls

Authentication drift is fundamentally a change management problem. It occurs because the email infrastructure changes — services are added, configurations are updated, infrastructure is migrated — without corresponding updates to authentication records. The prevention solution is process controls that ensure authentication implications are considered whenever email-related infrastructure changes occur.

The minimal process control: a change checklist for any addition or modification of email-sending services. Before any service is authorised to send email from the organisation's domain, verify: the service's sending IPs are covered by the SPF record (or will be added before the service goes live), the service signs with DKIM using a key that is published in DNS (or will be configured before go-live), and the DMARC aggregate report's expected sources list has been updated to include the new source IP range. This checklist — taking 15 minutes to complete — prevents the overwhelming majority of authentication drift events.

The quarterly authentication audit — reviewing all active sending sources, SPF lookup depth, DKIM selector validity, and DMARC alignment pass rates — catches any drift that the change checklist missed and provides a regular reset to a known-good authentication state. The audit takes 2–4 hours quarterly and produces a documented baseline that serves as the reference point for the next audit period.

Organisations that implement both the change checklist and the quarterly audit consistently find that their authentication pass rates remain above 99% across all sources, their SPF lookup counts stay within the 10-lookup limit, and their DMARC aggregate reports show only expected sources with consistently high alignment pass rates. The investment in these two process controls — perhaps 10 hours per year total — eliminates the category of deliverability problems caused by authentication drift, which is one of the most common root causes of unexplained inbox placement deterioration in mature sending programmes.

SPF Flattening: Mechanics and Trade-offs

SPF flattening is the process of converting include: references in the SPF record to explicit ip4: and ip6: IP address lists. The purpose is to reduce the DNS lookup count — a flattened SPF record that lists IP addresses directly requires zero lookups for those addresses, compared to multiple lookups for the same IPs accessed through nested include: chains. Flattening can reduce a 12-lookup SPF record to a 0-lookup record, eliminating the lookup limit as a constraint entirely.

The trade-off: flattened SPF records require maintenance. When a sending service changes its sending IP addresses — which cloud-based services do periodically — the flattened SPF record continues to authorise the old IPs and does not authorise the new ones until manually updated. Include: references, by contrast, automatically pick up IP changes when the service updates their published SPF include record. An organisation that flattens SPF records but does not maintain them ends up with a SPF record that is both within the lookup limit and incorrectly authorised — potentially worse than the over-limit record it replaced, because the over-limit record at least attempts to authorise the correct services.

The correct flattening approach: use a SPF management tool that automatically re-flattens the record when upstream include: records change. Several services (dmarcian, Valimail, AutoSPF) provide dynamic SPF flattening that resolves all includes to IP lists on a schedule (daily or hourly) and publishes the updated flat record to DNS automatically. This approach combines the lookup efficiency of flattening with the IP coverage maintenance of include: references, eliminating both the lookup limit problem and the maintenance burden.

DKIM Alignment Failure: Domain Mismatch Scenarios

DMARC alignment failure occurs when the domain in the DKIM signature (the d= tag) does not align with the From: domain in the message header. Alignment failure is one of the most common forms of authentication drift because it is introduced by well-intentioned service configuration changes that don't account for the DMARC alignment requirement.

The most common mismatch scenario: a marketing platform or ESP that signs messages with its own domain (mailchimp.com, salesforce.com, hubspot.com) rather than the sender's domain. A message from: newsletter@yourdomain.com signed by d=mcsv.net fails DMARC alignment because neither SPF alignment nor DKIM alignment passes — the DKIM signature is from the ESP's domain, not the sender's domain. The solution is to configure the ESP to sign with the sender's domain using a CNAME-delegated signing key — the ESP's infrastructure signs messages with a key published under yourdomain.com's DNS.

A subtler mismatch: a company with multiple domains (yourdomain.com for marketing, mail.yourdomain.com for transactional) where DMARC uses relaxed alignment. Under relaxed alignment, mail.yourdomain.com DKIM signatures align with the yourdomain.com DMARC policy because they share the organisational domain. But if the DMARC policy is on mail.yourdomain.com rather than yourdomain.com, relaxed alignment may not work as expected — the alignment rules require the organisational domain match, not any ancestor-descendant relationship. Verifying exactly where the DMARC record is published and confirming that alignment rules match the actual domain structure of the sending programme prevents this class of mismatch.

Recovering from Severe Authentication Drift

When authentication drift has been present for long enough to produce DMARC failures at scale — visible in Postmaster Tools as declining authentication pass rates and potentially in declining domain reputation — the recovery process must be sequenced carefully to avoid making the situation worse while fixing it.

The recovery sequence: (1) conduct the full authentication audit to identify all failure sources; (2) fix SPF first — this is the fastest to implement (DNS change propagates within minutes) and often resolves the largest share of failures; (3) fix DKIM signing for uncovered sources next — this requires configuration changes at the sending services themselves, which may take longer if they involve third-party platforms; (4) fix DMARC alignment for misaligned sources last — this requires coordinating with service providers to configure domain-aligned DKIM signing; (5) monitor DMARC aggregate reports daily during and after the fix sequence, confirming each source's alignment pass rate recovers as the fix is applied.

Do not accelerate DMARC policy to p=reject while authentication failures are still present. A DMARC policy escalation during the fix sequence will immediately affect delivery of messages from sources that have not yet been fixed, potentially causing delivery disruption to legitimate mail streams. Wait until all sources show 100% alignment pass rate in DMARC aggregate reports before escalating DMARC policy. The conservative sequencing — fix first, escalate second — consistently produces better outcomes than the "fix and escalate simultaneously" approach that is tempting but counterproductive.

Authentication Drift in Multi-Brand and Agency Environments

Multi-brand organisations and agencies managing email for multiple clients face authentication drift risk at scale — the same mechanisms operate across every client account, but each account has its own DNS configuration, its own set of sending services, and its own change history. A client that adds a new marketing automation tool without notifying the email infrastructure team introduces SPF drift into their account without any change occurring in the shared infrastructure.

The solution in multi-account environments is a standardised authentication inventory: a document (or database) that lists, for each client account, the current SPF record and lookup count, all active DKIM selectors and their corresponding sending sources, the DMARC policy and RUA/RUF addresses, and the date of the last authentication audit. This inventory is reviewed quarterly as part of the regular audit cycle, and updated whenever a client adds or changes a sending service.

Automated DMARC aggregate report parsing across all client accounts is the most scalable approach to drift detection in multi-account environments. A single parsing pipeline that ingests reports for all accounts and alerts when any account shows new source IPs or declining alignment pass rates provides centralised monitoring without requiring per-account manual review. The alert then triggers a targeted investigation into the specific account and source, rather than requiring daily review of all accounts regardless of whether drift has occurred.

The Long-Term Authentication Health Metric

A single metric captures overall authentication health better than any individual check: the DMARC pass rate — the percentage of messages that achieve DMARC compliance (passing either DKIM or SPF alignment, or both). This metric, drawn from DMARC aggregate reports, integrates all authentication mechanisms into a single pass/fail outcome that reflects the deliverability impact of authentication state.

A healthy sending programme maintains a DMARC pass rate above 99% consistently. A rate between 97–99% indicates minor authentication coverage gaps that are not yet causing delivery problems but should be investigated. A rate below 97% indicates authentication failures at a volume that is likely affecting deliverability and requires immediate investigation. A rate below 95% is a P1 incident regardless of whether current delivery rates appear normal, because the authentication failures are producing negative reputation signals that will affect inbox placement within weeks.

Tracking this metric monthly — graphed over time to show trends — provides the most compact view of authentication health. A DMARC pass rate that has been stable at 99.7% for 8 months and drops to 98.1% in the current period is a drift signal even though 98.1% is still "good" in absolute terms. The trend is the diagnostic, not the absolute value. Early detection and investigation of trends prevents the accumulation of authentication failures that eventually cross the threshold into delivery impact.

Building this metric tracking into the standard deliverability monitoring stack — alongside Gmail Postmaster Tools domain reputation and spam rate, Microsoft SNDS IP status, and accounting log delivery rate — creates a complete monitoring system that catches the authentication dimension of deliverability health before it produces the delivery-rate declines or reputation changes that more reactive monitoring would detect. Authentication drift caught at 1.6% DMARC failure rate is corrected with a DNS change and a configuration update. Authentication drift discovered at 8% DMARC failure rate — after inbox placement has already declined — requires the same fixes plus weeks of recovery time for the reputation signals to normalize. The monitoring investment that enables early detection consistently produces better outcomes at lower remediation cost.

Authentication drift is ultimately a maintenance discipline problem masquerading as a technical problem. The technical fixes — SPF flattening, DKIM key rotation, DMARC alignment configuration — are straightforward once the drift is identified. The challenge is identifying it early, before it affects delivery, and preventing it from re-accumulating after it is fixed. Programmes that establish the change checklist process, the quarterly audit cadence, and the DMARC pass rate tracking metric eliminate authentication drift as a recurring deliverability problem category. Those that address it reactively — fixing it when delivery problems appear, then allowing it to accumulate again until the next incident — spend disproportionate time on a preventable problem class while missing the proactive attention that produces genuinely excellent long-term deliverability outcomes.

In the ecosystem of email deliverability management, authentication drift occupies a specific position: it is a preventable problem that is common, often invisible until late, and entirely within the sender's control to fix. Unlike list quality problems that depend on acquisition practices outside the infrastructure team's scope, or ISP policy changes that are external constraints, authentication drift is caused by internal changes to internal systems. The domain owner controls every authentication record, every signing key, every SPF include. Owning this control fully — monitoring it continuously, maintaining it through change management, auditing it regularly — is the professional standard for email infrastructure management that produces the authentication reliability that both ISPs and deliverability outcomes require.