SMTP Authentication Mechanisms Compared: SPF, DKIM, DMARC, and ARC

• October 2022
• Engineering Memo · External Release

Email authentication is not a single mechanism but a layered system of four distinct protocols — SPF, DKIM, DMARC, and ARC — each designed to address a specific authentication problem that the others do not fully solve. Understanding what each mechanism does, what it does not do, and how they interact is the foundation for configuring authentication correctly and for diagnosing authentication failures when they occur.

This note compares the four mechanisms systematically, documents the configuration decisions that determine whether they work together effectively, and identifies the specific failure modes that arise when they interact incorrectly.

SPF: Authorising Sending IPs

Sender Policy Framework (SPF) addresses the question: is this IP address authorised to send email on behalf of this domain? The SPF record for a domain (a DNS TXT record at the root domain) lists the IP addresses and IP ranges that are authorised to send email with that domain in the SMTP MAIL FROM command (the envelope sender). A receiving server queries the SPF record for the MAIL FROM domain, checks whether the sending IP appears in the authorised list, and returns a pass (IP is authorised), fail (IP is not authorised and -all is specified), softfail (IP is not authorised but ~all is specified), or neutral (no policy specified) result.

SPF limitations: SPF only checks the envelope sender (MAIL FROM) domain, not the From: header domain that recipients see. A message with From: brand@brand.com in the header can pass SPF if the MAIL FROM uses an ESP domain (brand=bounces@esp.com) whose SPF record includes the sending IP. This is why SPF alone is insufficient for brand protection — it authenticates the bounce address domain, not the visible From: address. SPF also fails completely when a message is forwarded, because the forwarding server's IP is not in the original sender's SPF record.

The SPF record design constraint: each SPF record allows a maximum of 10 DNS lookups during evaluation. Records with more than 10 include: or exists: mechanisms that require DNS lookups produce an SPF permerror, which many ISPs treat as an SPF fail. For programmes using multiple ESPs and sending services, each with their own include: mechanism in the SPF record, the 10-lookup limit is a real operational constraint that requires careful SPF record design and periodic lookup count auditing.

DKIM: Cryptographic Message Signing

DomainKeys Identified Mail (DKIM) addresses the question: was this specific message actually sent by the domain it claims, and has it been modified in transit? DKIM uses asymmetric cryptography — a private key on the sending MTA signs the message headers and body, and a corresponding public key published in DNS allows any receiving server to verify the signature. A valid DKIM signature proves both that the message was signed by the domain that published the public key, and that the signed portions of the message have not been altered since signing.

DKIM's role in the authentication stack is the most operationally important: Gmail uses DKIM signing domain (d= value) as the primary domain to which it attributes reputation signals. A message signed with brand.com's DKIM key contributes to brand.com's domain reputation at Gmail, regardless of which IP sent it. This makes DKIM the foundational authentication mechanism for domain reputation building — without correct DKIM configuration, ISPs cannot accurately attribute reputation signals to the intended domain.

DKIM survives email forwarding (when the message body is not modified by the forwarder), unlike SPF. A message forwarded from recipient A to recipient B maintains its valid DKIM signature as long as the forwarder does not modify the signed portions of the message. This forwarding survivability makes DKIM the preferred authentication mechanism for DMARC alignment when forwarding is a concern for the programme's audience.

DMARC: Alignment and Policy Enforcement

Domain-based Message Authentication, Reporting, and Conformance (DMARC) addresses the question: is the authenticated sender identity aligned with the visible From: header domain, and what should the receiving server do when it is not? DMARC requires that at least one of SPF or DKIM passes alignment — the authenticated domain must match the From: header domain (or be a subdomain of it under relaxed alignment). When neither passes alignment, the DMARC policy (p=none, p=quarantine, or p=reject) determines the disposition.

DMARC's most operationally important feature for infrastructure management is its aggregate reporting (rua= tag). DMARC aggregate reports, sent daily by major ISPs, show every message received claiming the monitored From: domain, the sending IP, and the SPF and DKIM authentication and alignment results for each source. This reporting function makes DMARC an infrastructure monitoring tool as much as a policy enforcement tool — the aggregate reports reveal all sending sources for a domain, their authentication status, and any sources attempting to spoof the domain.

DMARC policy progression: the standard deployment path is p=none (monitoring mode — collect reports, no enforcement action), then p=quarantine (failed alignment messages routed to spam folder), then p=reject (failed alignment messages rejected outright). The progression should be gated on aggregate report review: only advance to the next policy level when aggregate reports confirm that all legitimate sending sources are passing alignment. Advancing to p=reject before all legitimate sources are confirmed passing produces rejected legitimate messages — which is a deliverability incident rather than a security improvement.

ARC: Authenticated Received Chain

Authenticated Received Chain (ARC, RFC 8617, 2019) addresses the authentication chain break problem: when an intermediary (mailing list processor, forwarding service, security gateway) modifies a message, it invalidates the original DKIM signature and may change the sending IP such that SPF fails. The message arrives at the final destination failing both DKIM and SPF, which may cause DMARC to fail even though the original sender's authentication was valid.

ARC allows each intermediate handler to add a cryptographic seal attesting to the authentication status of the message when it was received, before any modifications. The final receiving server can evaluate the ARC chain — a sequence of seals from each intermediary in the message path — to determine whether the original message was authenticated, even when the current authentication state has been broken by intermediary processing.

ARC is most relevant for programmes whose recipients use mailing lists or forwarding services. Gmail, Microsoft, and several other major ISPs implemented ARC verification by 2022, allowing ARC-sealed messages from reputable intermediaries to receive better treatment than unauthenticated messages. For direct sends from the sender's infrastructure to the recipient's ISP (the most common sending pattern for commercial email), ARC is not relevant — it is designed for indirect sending paths with intermediaries.

The Authentication Stack in Practice: Getting All Four Right

For a commercial email programme sending directly from its own infrastructure, the correct authentication configuration is: SPF record at each sending domain that includes all authorised sending IPs; DKIM signing configured in PowerMTA for each sending domain with RSA-2048 private keys and correctly formatted public key DNS records; DMARC record at each sending domain at p=none initially with rua= reporting to the programme's DMARC report processor; and DMARC policy advancement to p=quarantine or p=reject once aggregate reports confirm all legitimate sources pass alignment.

The DKIM signing domain (d= value) must match or be a parent domain of the From: header domain for DMARC alignment. This alignment requirement is the most common authentication configuration error: a programme that uses an ESP's shared sending infrastructure may have DKIM signing from the ESP's domain (esp.com) while the From: header shows brand.com. The DKIM signature is valid but does not align with the From: domain, so DMARC fails — and the message may be quarantined or rejected when DMARC is enforced, and brand.com's domain reputation is not built by the DKIM-authenticated signals.

The resolution for DMARC alignment with ESP infrastructure: configure the ESP to sign with the programme's domain (most ESPs support custom DKIM signing with the programme's own keys), or migrate to infrastructure where the programme controls the DKIM signing. The second path — own infrastructure with own DKIM signing — is the more durable solution: it provides complete authentication control, correct domain reputation attribution, and independence from ESP authentication configuration decisions.

The four-layer authentication stack — SPF authorising IPs, DKIM providing cryptographic message authentication, DMARC enforcing alignment and providing reporting, and ARC preserving authentication through forwarding chains — is the complete authentication architecture that major ISPs expect from professional email senders. Each layer plays a specific role that the others do not fully cover. Implementing all four correctly is the authentication investment that makes every other deliverability investment — IP warming, list quality management, monitoring — operate on a solid authentication foundation rather than a fragmented or partially-configured one.

Common Authentication Misconfigurations and Their Symptoms

SPF permerror from too many lookups. Symptom: DMARC aggregate reports show SPF permerror for some sending sources; some ISPs report SPF errors in message headers. Cause: SPF record contains more than 10 DNS lookup mechanisms. Investigation: audit the SPF record with an SPF checker tool (MXToolbox, dmarcanalyzer.com) that counts lookups. Resolution: flatten the SPF record (replace include: mechanisms with the specific IP ranges they expand to), reducing the lookup count below 10. Note that SPF flattening requires maintenance when ESP IP ranges change.

DKIM body hash mismatch. Symptom: Gmail Authentication-Results header shows dkim=fail (body hash did not verify). Cause: a mail relay in the delivery path modified the message body after DKIM signing. Investigation: review the Received: header chain to identify intermediate relays; test with direct delivery bypassing the relay. Resolution: configure the intermediate relay to not modify signed messages, or use relaxed canonicalization (relaxed/relaxed) in PowerMTA's DKIM signing configuration to tolerate minor body modifications.

DMARC alignment failure with valid DKIM. Symptom: DMARC aggregate report shows DKIM pass but dkim_aligned fail. Cause: DKIM d= domain does not match or is not a parent domain of the From: header domain. Investigation: check the d= value in the DKIM-Signature header of a test message vs the From: header domain. Resolution: configure the signing VMTA to use the From: header's domain as the d= value, or configure relaxed DMARC alignment and ensure the d= domain is a parent of the From: domain.

DMARC p=reject causing legitimate message rejections. Symptom: bounce reports show 550 5.7.1 rejections from ISPs citing DMARC policy; some campaign sends are completely rejected. Cause: DMARC policy was advanced to p=reject before all legitimate sending sources were confirmed to pass alignment. Resolution: temporarily downgrade to p=quarantine, review aggregate reports to identify the sources generating alignment failures, configure those sources for DKIM alignment, and verify with aggregate report review before returning to p=reject.

SPF softfail treated as fail by strict ISPs. Symptom: some ISPs reject messages with SPF ~all result; DMARC aggregate reports show SPF softfail for legitimate senders. Cause: SPF record ends with ~all (softfail) rather than -all (fail) for unlisted IPs. Investigation: check whether the ISP's rejection message cites SPF softfail specifically. Resolution: if all legitimate sending sources are in the SPF record, change the qualifier from ~all to -all. This produces a harder fail for unlisted IPs (which is correct if all legitimate IPs are listed) and removes the softfail ambiguity that some strict ISPs treat as a rejection basis.

The Authentication Audit Checklist

A complete authentication audit verifies all four layers across all sending domains. The checklist:

SPF audit: retrieve the SPF record for each sending domain with dig TXT domain.com; count the DNS lookups using an SPF checker tool; verify that all authorised sending IPs are included; confirm the record ends with a qualifier (-all or ~all); verify that subdomains used for sending have their own SPF records or inherit correctly from the parent domain.

DKIM audit: retrieve the DKIM TXT record for each active selector with dig TXT selector._domainkey.domain.com; verify the public key is correctly formatted (RSA-2048 keys require multiple quoted strings); send a test message and check the Authentication-Results header for dkim=pass; verify the d= value in the DKIM-Signature matches the From: domain for DMARC alignment; check the key generation date and schedule rotation if approaching 12 months.

DMARC audit: retrieve the DMARC record with dig TXT _dmarc.domain.com; verify the rua= tag points to a functional report receiver; review the most recent aggregate reports for authentication and alignment status of all sending sources; confirm that the current policy (none/quarantine/reject) is appropriate given the aggregate report data.

The authentication audit takes 2-3 hours for a programme with 2-3 sending domains and produces a clear picture of whether the authentication stack is correctly configured and complete. Running the audit annually — or after any infrastructure change that affects sending sources, IP addresses, or domains — maintains the authentication integrity that DMARC policy enforcement and domain reputation attribution depend on. Authentication drift (configurations that were correct at deployment but have become incorrect due to infrastructure changes) is the most common cause of authentication failures in established programmes. The annual audit catches drift before it produces reputation or deliverability consequences.

The four authentication mechanisms — SPF, DKIM, DMARC, and ARC — are not competing alternatives but complementary layers of a complete authentication architecture. Each solves a specific problem; none is sufficient alone. Implementing all four correctly, verifying them with the audit checklist, and maintaining them with the annual review discipline is the authentication investment that makes every reputation signal correctly attributed, every domain clearly authenticated, and every inbox placement decision based on accurate sender identity information. It is the foundational investment that all other deliverability work depends on to produce its intended results.

How ISPs Use Authentication Results

Authentication results influence ISP inbox placement decisions through several distinct channels. Understanding how each ISP uses authentication data makes clear why the authentication investment produces the inbox placement returns it does.

Gmail: DKIM d= domain is the primary attribution identifier for domain reputation. Gmail Postmaster Tools domain reputation data is specific to the DKIM signing domain, not the From: header domain (though they should be the same for aligned senders). DMARC alignment is required for DKIM signals to accumulate on the intended domain. SPF is used as a secondary signal; Gmail does not require SPF for inbox placement but uses SPF failure as a negative signal for new or low-reputation senders. Senders without DKIM or with consistently failing DKIM are treated as unauthenticated and face more aggressive spam filtering.

Microsoft (Outlook/Hotmail): SPF is more heavily weighted by Microsoft than by Gmail. Microsoft's Exchange Online Protection (EOP) uses SPF pass as a positive trust signal for inbox placement; SPF fail or softfail is a negative trust signal. DKIM is verified and used for reputation attribution. DMARC enforcement is applied according to the published policy. Microsoft's SNDS programme provides IP reputation data that complements domain-level authentication signals in EOP's filtering decisions.

Yahoo: Yahoo requires valid SPF or DKIM for its FBL programme registration and uses both for inbox placement decisions. Yahoo's FBL complaint data is attribution to the sending domain (for DKIM-signed messages) or sending IP (for unsigned messages). Senders with DKIM signing receive cleaner FBL complaint attribution and more reliable reputation signal processing than unsigned senders.

The ISP-specific weighting differences explain why DKIM is the most important authentication mechanism for Gmail deliverability (reputation attribution), while SPF takes on additional importance for Microsoft deliverability (trust signal for EOP). A complete authentication stack that includes both, with DMARC alignment enforcing their consistency, covers the full authentication requirements of the major ISP landscape without requiring ISP-specific authentication configuration choices.

Authentication configuration is one of the most stable investment categories in email infrastructure: the mechanisms change slowly (SPF was standardised in 2014, DKIM in 2011, DMARC in 2015, ARC in 2019), the configuration once correct remains correct until infrastructure changes, and the return on correct authentication configuration compounds indefinitely as reputation accumulates on the correctly authenticated sender identity. Get it right once, audit it annually, and maintain it consistently — the infrastructure foundation that authentication provides supports every other deliverability investment for the lifetime of the programme.

SPF to authorise. DKIM to sign. DMARC to enforce alignment and report. ARC to preserve through forwarding. Four mechanisms, four distinct roles, one complete authentication architecture. Configure all four, verify them with the audit, and maintain them with the annual review. The authentication foundation this provides is what makes reputation-based deliverability management work correctly -- and what protects the brand from spoofing while building the reputation that inbox placement depends on.

Authentication in the 2024 Context

By early 2024, Google and Yahoo formally required DKIM authentication and DMARC at p=none for all bulk senders to their platforms. These requirements formalised what had been operational best practice for years -- senders without DKIM authentication were already experiencing deliverability disadvantages long before the formal requirement was announced. The formal requirement converted authentication from a best practice into a baseline eligibility requirement for inbox delivery.

The 2024 authentication requirements from Google and Yahoo also required compliance with one-click unsubscribe (RFC 8058 List-Unsubscribe-Post) for bulk senders, making authentication and unsubscribe compliance linked in the same enforcement framework. This integration reflects the ISP perspective that authentication, unsubscribe compliance, and spam rate management are all components of a single responsible sender profile rather than independent technical requirements.

Programmes that implemented the full authentication stack (SPF, DKIM, DMARC) as a standard practice before the 2024 requirements experienced no disruption from the formal enforcement. Programmes that had deferred authentication configuration until it was required faced a compressed timeline for implementing changes that typically take 4-6 weeks to complete correctly -- DKIM key generation, DNS publishing, PowerMTA configuration, DMARC deployment and aggregate report review -- which produced deliverability disruption during the implementation period. The lesson from the 2024 enforcement cycle is the same as the general lesson from all authentication investment: implement correctly before it is required, and the requirement becomes an operational checkpoint rather than an emergency project.