SPF Record Flattening: Why Your SPF Is Failing the 10-Lookup Limit and How to Fix It

SPF's 10 DNS lookup limit is, in my experience, the most common authentication configuration error in production email infrastructure — and the hardest one to detect because it fails silently in the worst possible way. When an SPF evaluation hits the 10-lookup limit during processing, the result is not a clean "SPF fail" that gets logged and can be debugged. The result is "SPF permerror" — a permanent error that, depending on how the receiving mail server interprets it, may cause DMARC to fail, may cause the email to be treated as unauthenticated, and may do so without any error message appearing in the sending MTA's logs. The email was sent, the MTA reports delivery, and the authentication failure is invisible until a deliverability incident reveals it.

I have debugged this problem at three separate organisations in the past year alone — each time, the root cause was SPF lookup count creep that happened gradually as new sending services were added to the infrastructure and nobody went back to audit the total lookup count. This guide documents how to find the problem before it finds you, and how to fix it when it does.

The 10-Lookup Limit: Why It Exists and Why It Breaks

RFC 7208 (the SPF specification) states: "SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the 'include' mechanism or the 'redirect' modifier." The limit exists to prevent SPF verification from becoming a vector for DNS denial-of-service attacks — an SPF record that chains through 50 include: directives would require 50 DNS queries from every receiving mail server that checks SPF, which at scale represents a meaningful amplification attack against DNS infrastructure.

The limit breaks modern email deployments because the email service ecosystem has fragmented dramatically since SPF was designed. A modern company might send email through Google Workspace (include:_spf.google.com, which itself chains to multiple records), a marketing platform like Klaviyo (include:klaviyomail.com), a transactional provider like Postmark (include:spf.mtasv.net), a customer success platform like Intercom (include:mail.intercom.io), a support tool like Zendesk, Salesforce for sales email, and several legacy internal sending systems. Each include: directive consumes at least one lookup. The lookup count adds up faster than most organisations track.

The specific failure mode depends on the receiving server's interpretation of the SPF permerror result. RFC 7208 says that permerror "indicates that the domain's published records could not be correctly interpreted" — but it does not mandate what the receiving server should do with a permerror result. Some servers treat permerror as SPF neutral (neither pass nor fail). Some treat it as SPF fail. Some treat it as a hard authentication failure for DMARC purposes. The inconsistency in permerror handling is what makes the lookup limit failure so damaging — the same SPF lookup-limit violation may cause DMARC failure at some ISPs, neutral SPF result at others, and no visible problem at still others. This inconsistency makes it very difficult to detect through standard monitoring.

Why SPF Lookup Limit Failures Are Silent and Destructive

Here is what makes the SPF lookup limit failure particularly insidious: the sending MTA has no knowledge of what happens during SPF evaluation at the receiving end. The sending MTA submits the email, receives a 250 OK acceptance response, records "delivery success," and moves on. The SPF evaluation happens entirely within the receiving server's processing pipeline, after the 250 OK is sent. By the time the receiving server discovers that SPF evaluation hits the lookup limit (which can require up to 10 DNS queries to determine), the SMTP session is already closed.

The result: your MTA logs show 100% delivery success. Your ESP dashboard shows 100% delivery rate. Your DMARC aggregate reports — if you are reviewing them — show SPF failures for a fraction of sends, without any obvious correlation to a specific sending source. Gmail Postmaster Tools may show degrading domain reputation without a visible cause in the spam rate or authentication charts. Recipients at specific ISPs report not receiving email, while recipients at other ISPs receive it normally. The pattern is confusing until you think to check SPF lookup count, at which point the answer is usually obvious.

The organisations most at risk: companies that have grown rapidly and added new SaaS tools with email components without auditing the cumulative SPF lookup count; companies that use a complex ESP stack with multiple sending services each requiring their own include: directive; and companies that have experienced mergers or acquisitions that combined two SPF configurations into one without rationalisation.

Counting Your Current Lookups: The Diagnostic Script

#!/usr/bin/env python3
"""
SPF Lookup Counter — counts all DNS lookups required to evaluate an SPF record
Including nested includes and redirects
Run: python3 spf_count.py yourdomain.com
"""

import dns.resolver
import sys

def count_spf_lookups(domain, depth=0, visited=None):
    """Recursively count DNS lookups in SPF chain"""
    if visited is None:
        visited = set()
    if domain in visited:
        return 0, []
    visited.add(domain)

    try:
        answers = dns.resolver.resolve(domain, 'TXT')
    except Exception as e:
        return 1, [f"  LOOKUP FAILED: {domain} ({e})"]

    spf = None
    for record in answers:
        txt = str(record).strip('"')
        if txt.startswith('v=spf1'):
            spf = txt
            break

    if not spf:
        return 0, [f"  NO SPF: {domain}"]

    count = 0
    details = [f"{'  ' * depth}SPF {domain}: {spf[:80]}"]
    
    for term in spf.split():
        # These mechanisms each consume a DNS lookup:
        if term.lower() in ('a', 'mx', 'ptr') or            term.lower().startswith('a:') or            term.lower().startswith('mx:') or            term.lower().startswith('ptr:'):
            count += 1
            details.append(f"{'  ' * (depth+1)}+1 lookup: {term}")
        elif term.lower().startswith('include:'):
            nested_domain = term[8:]
            count += 1  # The include lookup itself
            details.append(f"{'  ' * (depth+1)}+1 lookup: include:{nested_domain}")
            sub_count, sub_details = count_spf_lookups(nested_domain, depth+2, visited)
            count += sub_count
            details.extend(sub_details)
        elif term.lower().startswith('redirect='):
            nested_domain = term[9:]
            count += 1
            details.append(f"{'  ' * (depth+1)}+1 lookup: redirect={nested_domain}")
            sub_count, sub_details = count_spf_lookups(nested_domain, depth+2, visited)
            count += sub_count
            details.extend(sub_details)
        elif term.lower().startswith('exists:'):
            count += 1
            details.append(f"{'  ' * (depth+1)}+1 lookup: {term}")

    return count, details

if __name__ == '__main__':
    domain = sys.argv[1] if len(sys.argv) > 1 else input("Domain: ")
    total, details = count_spf_lookups(domain)
    
    for line in details:
        print(line)
    
    print(f"
TOTAL DNS LOOKUPS: {total}/10")
    if total > 10:
        print(f"⛔ EXCEEDS LIMIT by {total - 10} lookups — SPF will produce permerror")
    elif total > 8:
        print(f"⚠️  APPROACHING LIMIT — review before adding more services")
    else:
        print(f"✅ Within limit")

Install dns.python with pip install dnspython and run this against your sending domain. The output shows every include chain, every lookup consumed by each mechanism, and the total. If you are at 9 or 10, you are one new ESP away from a lookup limit failure. If you are already above 10, you have an active problem that may be causing authentication failures right now.

Which SPF Mechanisms Count Against the Limit

Not all SPF mechanisms consume DNS lookups. Understanding which mechanisms count is essential for optimising SPF records:

Manual Flattening: How to Do It Without a Service

SPF flattening is the process of replacing include: mechanisms with the explicit ip4: and ip6: addresses that the includes resolve to — eliminating the DNS lookups those includes require. Instead of:

# BEFORE flattening — 12 DNS lookups (over limit):
"v=spf1 include:_spf.google.com include:spf.protection.outlook.com
  include:spf.mtasv.net include:klaviyomail.com include:sendgrid.net
  include:spf.intercom.io ~all"

# AFTER manual flattening — 0 DNS lookups for the core record:
"v=spf1
  ip4:209.85.128.0/17 ip4:64.233.160.0/19 ip4:66.102.0.0/20
  ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16
  ip4:173.194.0.0/16 ip4:207.126.144.0/20 ip4:216.58.192.0/19
  ip4:216.239.32.0/19
  ip4:40.92.0.0/15 ip4:40.107.0.0/16 ip4:52.100.0.0/14
  ip4:104.47.0.0/17
  ip4:65.55.88.0/24 ip4:216.32.180.0/23
  ip4:198.2.130.0/23 ip4:198.2.132.0/22 ip4:209.85.128.0/17
  ~all"

To do this manually: (1) Use the SPF counter script to identify all include: chains. (2) For each include, query the DNS records recursively until you reach the actual ip4: and ip6: addresses. (3) Collect all the ip4: and ip6: CIDR ranges from all includes. (4) Replace all the include: mechanisms with the collected ip4:/ip6: addresses directly in your SPF record.

The critical problem with manual flattening: the IP addresses behind each include: can change without notice. Mailchimp, Klaviyo, Google, and every other ESP occasionally adds or changes the IP addresses in their SPF records. When they do, your flattened SPF record becomes incorrect — it still lists the old IPs but not the new ones. Email authenticated against the new IPs will fail SPF. This is why manual flattening requires a monitoring process to detect when the underlying include: records change.

#!/usr/bin/env bash
# SPF change monitor — run weekly via cron to detect include changes
# Compares current include resolution against a stored baseline

DOMAIN="yourdomain.com"
BASELINE_FILE="/var/log/spf-baseline.txt"

# Extract current IPs resolved from all includes
current_ips() {
    python3 - "$DOMAIN" << 'EOF'
import dns.resolver, sys

def get_spf_ips(domain, visited=None):
    if visited is None:
        visited = set()
    if domain in visited:
        return set()
    visited.add(domain)
    
    ips = set()
    try:
        answers = dns.resolver.resolve(domain, 'TXT')
        for record in answers:
            txt = str(record).strip('"')
            if not txt.startswith('v=spf1'):
                continue
            for term in txt.split():
                if term.startswith('ip4:') or term.startswith('ip6:'):
                    ips.add(term)
                elif term.startswith('include:'):
                    ips |= get_spf_ips(term[8:], visited)
    except:
        pass
    return ips

for ip in sorted(get_spf_ips(sys.argv[1])):
    print(ip)
EOF
}

if [ ! -f "$BASELINE_FILE" ]; then
    current_ips > "$BASELINE_FILE"
    echo "Baseline created: $(wc -l < $BASELINE_FILE) IP ranges"
    exit 0
fi

DIFF=$(diff "$BASELINE_FILE" <(current_ips))
if [ -n "$DIFF" ]; then
    echo "SPF CHANGE DETECTED for $DOMAIN:"
    echo "$DIFF"
    echo "Update your flattened SPF record and baseline immediately"
    # Send alert to ops team
    current_ips > "$BASELINE_FILE"
fi

Automatic SPF Flattening Services: When to Pay for Help

Several commercial services automate SPF flattening and handle the maintenance problem — monitoring the ESP include records for changes and automatically updating the flattened SPF record when they change:

AutoSPF.com: Replaces your existing SPF record with a DNS record that points to their service, which resolves dynamically to the current flattened IP list. Their infrastructure queries all your include: chains, caches the results, and serves the flattened IP list from a single DNS lookup. The SPF record in your DNS points to their service; their service does the heavy lifting. Pricing based on domains managed.

PowerDMARC SPF Flattener: Integrated into the PowerDMARC platform, which also provides DMARC reporting. If you are already using PowerDMARC for DMARC reporting, the SPF flattening feature may be included in your plan. The same automated monitoring and update approach as AutoSPF.

dmarcian SPF Surveyor: dmarcian provides SPF lookup analysis and recommendations for users of their DMARC reporting platform. Less fully automated than AutoSPF but provides excellent diagnostics and guidance for manual flattening.

When automatic flattening services are worth the cost: at 10+ include: mechanisms where manual maintenance becomes burdensome, in large organisations where DNS changes require change control processes (making it operationally difficult to update the SPF record every time an ESP changes its IP ranges), and in high-compliance environments where authentication failures have significant business consequences. For small operations with fewer than 6-7 include: mechanisms and internal DNS change control, manual flattening with the monitoring script is usually sufficient.

The Maintenance Problem: Why Flattening Is Not a One-Time Fix

The maintenance problem is the reason most SPF flattening approaches eventually break. ESPs change their sending IP ranges regularly — expanding capacity, migrating infrastructure, decommissioning old networks. When they do, they update their SPF include records. A flattened SPF record that replaced those includes with explicit IP addresses is now wrong — it lists old IPs and is missing the new ones.

Documented ESP SPF changes that have broken flattened records: Mailchimp expanded their sending infrastructure in 2024 and 2025, adding new IP ranges to their SPF record; senders with flattened records that did not include the new ranges saw authentication failures for email sent from the new infrastructure. SendGrid (now Twilio SendGrid) has made multiple changes to their SPF record as their infrastructure has evolved. Google's SPF record changes less frequently but has changed. Any ESP can change their IP ranges at any time, and they are under no obligation to notify individual senders who have flattened those records.

The operational minimum for maintaining a flattened SPF record: run the change detection script weekly (the bash script above), and update the flattened record within 48 hours of any detected change. The window between an ESP changing their IP ranges and SPF authentication failing for email sent from those new IPs is the time between the change and the next change detection run. Weekly detection means maximum 7 days of potential authentication failure for email from new ESP IPs — which is why organisations with high compliance requirements or large volumes should use an automatic flattening service that detects changes within hours rather than the monitoring interval.

Reducing Lookup Pressure: Structuring SPF to Avoid the Problem

Before flattening, consider whether structural changes can reduce lookup count without flattening:

Use subdomain SPF records: Instead of routing all sending through the root domain's SPF record, use subdomains with separate SPF records for different email streams. marketing.brand.com has its own SPF covering Klaviyo and Mailchimp. transactional.brand.com has its own SPF covering Postmark. support.brand.com has its own SPF covering Zendesk. The root domain's SPF only needs to cover the primary sending infrastructure. This approach works when the From: header address uses the subdomain — which requires configuring the relevant ESPs to send from the subdomain rather than the root domain.

Eliminate redundant mechanisms: Many SPF records contain include: directives from legacy services that are no longer used, or from services whose IP ranges are already covered by other includes in the record. Audit each include: mechanism against actual sending services — remove any that are no longer active. A quarterly review of "is this ESP still in use?" against the SPF record is a simple governance practice that prevents lookup count from growing unchecked.

Replace a/mx mechanisms with ip4:: If your SPF record uses a or mx mechanisms (which look up the domain's A or MX records), replace these with explicit ip4: addresses for the servers those mechanisms resolve to. The A or MX record for your domain presumably points to stable infrastructure — replace the DNS lookup with the explicit IP to eliminate the lookup consumption without requiring full record flattening.

SPF lookup limit management is an ongoing operational discipline, not a one-time fix. The organisations that handle it well run quarterly SPF audits (count current lookups, identify any new include: directives added without review), maintain automated change detection for flattened records, and have a documented process for adding new ESPs to the SPF configuration that includes assessment of lookup count impact before the change is deployed. The organisations that handle it poorly add new ESPs when needed, discover an authentication failure months later, and then spend time debugging a problem that the quarterly audit would have prevented.