SPF Record Complexity and Delivery Failure Risk

• March 2023
• Engineering Memo · External Release

SPF (Sender Policy Framework) is the email authentication standard that authorises which IP addresses may send email on behalf of a domain. Its design constraint — a maximum of 10 DNS lookup mechanisms per SPF evaluation — is straightforward in simple configurations but becomes operationally significant when a domain sends email through multiple services, each adding its own include: mechanism to the SPF record. When the total lookup count exceeds 10, SPF evaluation produces a "permerror" result, which many ISPs treat as equivalent to an SPF fail. The result: messages that should pass SPF authentication instead fail it, contributing to DMARC failures and reduced ISP trust signals.

This note documents how SPF permerror occurs, how to audit the current SPF record lookup count, the specific ESP configurations that most commonly cause the problem, and the techniques for resolving it while maintaining complete SPF coverage for all legitimate sending sources.

The 10-Lookup Limit: What Counts

The SPF specification (RFC 7208) limits SPF evaluation to 10 DNS lookups per SPF record. The mechanisms that consume lookups are: include: (each include consumes 1 lookup, plus all the lookups within the included record), a: (1 lookup per mechanism), mx: (1 lookup per mechanism), ptr: (1 lookup per mechanism, plus lookups for each PTR result), exists: (1 lookup per mechanism). The mechanisms that do NOT consume lookups are: ip4:, ip6:, and the final qualifier (-all, ~all, ?all).

A common misunderstanding: the 10-lookup limit counts recursively. If the SPF record includes include:_spf.google.com and Google's SPF record itself contains 3 include: mechanisms, those 3 inner lookups count toward the 10-lookup total, plus the 1 lookup for the outer include:_spf.google.com = 4 lookups total for that single include. This recursive counting is why adding multiple ESPs to a domain's SPF record can exceed the limit faster than expected — each ESP's SPF record may itself contain multiple include: mechanisms.

The mechanisms that commonly contribute the most lookups in commercial email configurations: Google Workspace sending (typically 1-3 lookups via include:_spf.google.com), Sendgrid (3-4 lookups via include:sendgrid.net), Mailchimp/Mandrill (2-3 lookups), Salesforce Marketing Cloud (3-5 lookups), and the programme's own sending infrastructure (1-2 lookups for direct IP ranges). A configuration using Google Workspace + Sendgrid + Mailchimp + Salesforce + own infrastructure reaches 10-17 lookups — well above the limit even before counting the mechanisms within each nested record.

Auditing the Current SPF Record

The SPF record audit is the first step in diagnosing permerror. Free tools including MXToolbox SPF Check, Dmarcanalyzer SPF Checker, and EasyDMARC SPF Checker evaluate a domain's SPF record and count the total number of DNS lookups it generates, including all recursive includes. These tools display the total lookup count and identify which mechanisms contribute the most lookups. Running the audit on the domain's SPF record produces the current lookup count and identifies the highest-priority mechanisms to address if the count exceeds 10.

The DMARC aggregate report is the operational confirmation that SPF permerror is occurring in production. If the aggregate report shows spf=permerror or spf=temperror for messages from the programme's own sending sources (rather than just for unknown sources), the SPF record has exceeded the lookup limit and is producing authentication failures for legitimate sends. Permerror in the aggregate report is the signal that the audit and resolution in this note are urgently needed.

The SPF audit should be run quarterly and any time a new sending service is added to the configuration. Adding a single new ESP to the SPF record may push a record that was previously at 9 lookups into permerror territory. Catching this before it affects production authentication requires checking the lookup count before the new include: is added to the production DNS record, not after delivery failures confirm the problem.

Resolving SPF Permerror: The Flattening Approach

SPF flattening resolves permerror by replacing include: mechanisms (each of which generates DNS lookups for the included record) with direct ip4: and ip6: mechanisms (which generate no lookups). Instead of including Sendgrid's SPF record with all its nested lookups, the programme directly lists the specific IP ranges that Sendgrid uses for its sending pool. The SPF record then contains only direct IP listings with no include: mechanisms — and therefore zero DNS lookups, well within the 10-lookup limit.

The flattening approach has one significant operational drawback: when an ESP changes its sending IP ranges (which occurs when they add new sending infrastructure), the programme's SPF record must be updated to include the new IP ranges. A programme using un-flattened include: mechanisms automatically picks up IP range changes when the ESP updates its own SPF record. A programme using flattened direct IP listings does not — it must manually update its SPF record when the ESP's IP ranges change, or accept temporary SPF failures for messages sent from the new IPs.

The practical resolution for most programmes: flatten the highest-lookup-count ESP include: mechanisms (typically Salesforce and Sendgrid, which consume the most lookups) while keeping lower-count include: mechanisms for services with stable, small IP ranges (Google Workspace's include:_spf.google.com is well-maintained and predictable). The hybrid approach reduces the total lookup count below 10 while minimising the maintenance burden by keeping include: mechanisms for only the most lookup-efficient services.

Several commercial services offer automated SPF flattening: EasyDMARC Managed SPF, AutoSPF, and dmarcdigests SPF Flattening maintain updated SPF records that automatically incorporate ESP IP range changes, solving the maintenance problem of manual flattening. These services replace the domain's SPF record with their managed record, which they keep current as ESPs update their IP ranges. The cost (typically €10-30/month) is well justified for programmes with multiple ESP integrations where manual flattening maintenance is operationally burdensome.

SPF record complexity is a maintenance debt that accumulates with each new sending service added to a programme's infrastructure. A domain that started with a simple SPF record and has added 5 ESPs over 3 years may not realise it has accumulated a permerror condition until DMARC aggregate reports reveal the authentication failures. The resolution -- auditing the lookup count, identifying the highest-lookup mechanisms, and applying the appropriate flattening strategy -- is a one-time configuration project that then requires only periodic maintenance as sending services are added or changed. Schedule the audit quarterly, keep the lookup count below 8 as a safety buffer, and SPF permerror will never become a production authentication problem.

The Temperror vs Permerror Distinction

SPF evaluation can produce two error results that are distinct from a simple fail. Permerror ("permanent error") occurs when the SPF record is syntactically invalid or exceeds the 10-lookup limit — the record cannot be evaluated correctly and produces a definitive error result. Temperror ("temporary error") occurs when a DNS lookup required by the SPF record fails due to a transient DNS resolution error — the DNS server was unavailable, returned a SERVFAIL, or timed out during SPF evaluation.

The operational significance differs: permerror is a configuration problem that requires SPF record correction to resolve. Temperror is a transient infrastructure event that typically self-resolves when the DNS issue clears. However, temperror can also be a symptom of SPF configuration problems: a record that causes excessive DNS lookups may trigger DNS timeout temperrors because the DNS resolution chain takes too long to complete. If temperror appears consistently in DMARC aggregate reports (not just occasionally), it may indicate an SPF complexity issue causing DNS timeouts rather than a transient DNS infrastructure problem.

Distinguishing permerror from temperror in aggregate reports: permerror appears consistently across all messages from affected sources; temperror appears intermittently across some messages from affected sources. A source that shows 100% SPF permerror has a configuration problem; a source that shows 5-15% SPF temperror has a DNS resolution intermittency (which may indicate configuration complexity causing DNS timeouts, or genuine DNS infrastructure instability). Monitoring the error type and frequency provides the diagnostic information needed to determine the correct resolution path.

ISP Treatment of SPF Permerror

ISPs vary in how strictly they treat SPF permerror. Gmail, as of 2023, does not reject or aggressively filter messages based on SPF permerror alone — if DKIM alignment passes, the message can still pass DMARC and receive normal delivery treatment. Yahoo and Microsoft are somewhat stricter: both use SPF permerror as a negative trust signal that may increase the probability of spam classification, particularly for senders without established High domain reputation. Some older or less sophisticated ISPs treat SPF permerror as equivalent to SPF fail and apply the same filtering response.

The practical implication: SPF permerror is tolerable for programmes with reliable DKIM alignment (DMARC passes via DKIM even when SPF permerrors), but it is a preventable authentication weakness that adds unnecessary risk. For senders with High Gmail domain reputation and reliable DKIM alignment, SPF permerror is unlikely to cause immediate inbox placement problems. For senders with Medium domain reputation, or those sending to ISPs that weight SPF more heavily, the permerror adds authentication risk that the flattening solution can eliminate with minimal effort.

The risk calculus: fix the permerror before it becomes a problem rather than after a reputation event forces the resolution. A programme that fixes its SPF record proactively spends 2-4 hours on the audit and flattening; a programme that waits for an inbox placement problem attributable to SPF authentication failures may spend weeks in reputation recovery investigation before the SPF permerror is identified as the contributing cause. The preventive approach is always less costly than the reactive one.

SPF Record Best Practices for Complex Sending Environments

For programmes managing SPF records across multiple sending domains with multiple ESPs, a set of operational best practices prevents permerror from recurring after the initial cleanup:

Maintain a sending source inventory. A spreadsheet listing every service that sends email on behalf of each sending domain, the SPF mechanism that authorises it, and the lookup count it contributes. When a new sending service is added, check the spreadsheet to determine the new total lookup count before adding the include: to the DNS record. If the new total would exceed 8 (leaving a safety buffer), apply flattening to an existing include: before adding the new one.

Prefer ip4: over include: for stable IP ranges. The programme's own MTA infrastructure sends from fixed IP ranges that do not change unless the hosting provider changes. These should use direct ip4: mechanisms rather than include: mechanisms pointing to SPF records at the infrastructure provider. This is the lowest-risk optimisation because it eliminates lookup overhead for the most stable and controllable sending source.

Audit on change, not on schedule alone. SPF record audits should be triggered by configuration change events (adding a new ESP, changing hosting providers, migrating to new sending IPs) as well as by the quarterly schedule. Configuration changes are the most common cause of new permerror conditions; catching the problem immediately after the change that caused it is more efficient than discovering it 3 months later in the quarterly audit.

Use the SPF record v=spf1 tag correctly. A domain should have exactly one SPF record (one TXT record starting with v=spf1). Multiple SPF records for the same domain produce a permerror during SPF evaluation. This is a common misconfiguration when SPF records are added to different DNS management interfaces — the IT team adds one, the ESP adds another during onboarding, and the domain ends up with two SPF records. Verify that only one SPF TXT record exists per domain with dig TXT domain.com and ensure all authorisations are consolidated into that single record.

SPF record management is infrastructure maintenance that requires the same discipline as any other infrastructure configuration: track the current state, understand the constraints, and manage changes proactively. The 10-lookup limit is the primary constraint; the per-domain single-record requirement is the secondary one. Both are manageable with an accurate sending source inventory and a quarterly audit discipline. Build the inventory, run the audits, and SPF permerror will never produce a production authentication failure that could have been caught and prevented in the audit.

The Maintenance Automation Opportunity

For programmes with engineering resources, the SPF record maintenance process can be partially automated. A scheduled script (weekly or monthly) that: (1) fetches the current SPF record for each monitored domain via DNS lookup, (2) counts the total lookup chain depth recursively, (3) compares against a configured warning threshold (8 lookups) and critical threshold (10 lookups), and (4) sends an alert when either threshold is exceeded — converts the SPF lookup count check from a manual quarterly exercise into an automated continuous monitoring practice.

The script implementation is straightforward in Python using the dnspython library: resolve the SPF record for the domain, parse the SPF mechanisms, recursively resolve any include: mechanisms, count all non-ip4/ip6 mechanisms encountered throughout the resolution chain, and return the total. A check scheduled via cron to run weekly adds SPF lookup count monitoring to the infrastructure health monitoring stack with approximately 4 hours of engineering investment.

For programmes without dedicated engineering resources, the quarterly manual audit using the free SPF checker tools described earlier provides adequate oversight when combined with the change-triggered audit discipline: always check the lookup count before adding a new include: mechanism to any production SPF record. The manual approach requires operational discipline rather than engineering investment — the commitment to always run the audit before a change, rather than adding the mechanism and checking later.

SPF and the Broader Authentication Picture

SPF record complexity and permerror are infrastructure concerns that sit at the intersection of the authentication stack and the operational management discipline. Permerror is a purely technical problem — a configuration error that produces a predictable authentication failure — but its operational impact (degraded ISP trust signals, potential DMARC failures) is a deliverability concern. The resolution requires both the technical fix (SPF record audit and flattening) and the operational discipline (quarterly audits and change-triggered checks) to prevent recurrence.

In the broader authentication context, SPF permerror is worth resolving for the same reason that all authentication is worth getting right: authentication is the foundation of reputation attribution, and reputation attribution is the foundation of inbox placement. A permerror that causes SPF to fail for messages that should pass is a leak in the authentication foundation — one that is entirely preventable with the audit and maintenance practices in this note. Eliminate the leak, maintain the foundation, and the authentication stack supports reputation building as it is designed to.

The programmes that never encounter SPF permerror as a production problem are those that maintain an accurate sending source inventory, run the SPF audit on every configuration change, and apply flattening proactively when the lookup count approaches the limit. These are not heroic operational practices — they are 30-minute quarterly exercises and 5-minute pre-change checks. The return on this modest investment: SPF authentication that consistently passes for all legitimate sends, DMARC compliance that does not depend entirely on DKIM alignment as a permerror fallback, and ISP trust signals that reflect the programme's actual authentication quality rather than a configuration oversight that looks identical to a deliberate authentication failure. Audit it, maintain it, and SPF will be an authentication asset rather than an authentication liability.

The 10-lookup limit is a 25-year-old design constraint that was adequate when most organisations sent from one or two services. Modern email sending environments -- with multiple ESPs, marketing platforms, CRM systems, and transactional providers -- can exceed this limit with entirely legitimate sending sources. Understanding the limit, auditing against it regularly, and applying the flattening solution when needed is the operational discipline that keeps SPF from becoming the authentication problem that quietly undermines the deliverability investment built on top of it.

Practical SPF Record Examples

A well-structured SPF record for a programme using Google Workspace for internal email, PowerMTA for bulk sending, and one ESP (Sendgrid) for transactional notifications might look like this:

v=spf1
  include:_spf.google.com
  ip4:203.0.113.1/24
  ip4:203.0.113.64/26
  include:sendgrid.net
  -all

In this example: include:_spf.google.com contributes approximately 3 lookups (the include itself plus two lookups within Google's record). The two ip4: mechanisms for the PowerMTA sending IPs contribute zero lookups. include:sendgrid.net contributes approximately 3-4 lookups. Total: approximately 6-7 lookups — within the 10-lookup limit with safety buffer.

If the programme adds Salesforce Marketing Cloud (which contributes 5 additional lookups via include:_spf.salesforce.com), the total becomes 11-12 lookups — over the limit. The resolution: flatten the Sendgrid include: by replacing include:sendgrid.net with the specific ip4: ranges that Sendgrid currently uses for the programme's account. This removes the 3-4 Sendgrid lookup chain and replaces it with zero-lookup ip4: mechanisms, reducing the total below 10 again.

The flattened record after Salesforce addition: v=spf1 include:_spf.google.com ip4:203.0.113.1/24 ip4:203.0.113.64/26 ip4:[sendgrid-ip-range-1] ip4:[sendgrid-ip-range-2] include:_spf.salesforce.com -all. Total lookups: approximately 8 — within the safe range. The Sendgrid IP ranges must be updated manually when Sendgrid changes them, but this is a periodic maintenance task rather than an ongoing operational burden. Quarterly audit catches any new Sendgrid IP ranges that need to be added.

SPF record management, done well, is configuration maintenance at its most straightforward: understand the constraint (10 lookups), know the current count (quarterly audit), and apply the resolution (flattening) before the count exceeds the limit. The programmes that manage it well spend a few hours per year on SPF maintenance. Those that do not spend unpredictable hours investigating authentication failures that could have been prevented with that few hours of proactive maintenance. The investment calculus is clear.

Audit the SPF record. Know the lookup count. Keep it below 8. Flatten when needed. Repeat quarterly. That operational discipline is all that stands between correct SPF authentication and the permerror that silently undermines it.