The Case for Real-Time Bounce Processing at the MTA Layer

• July 2022
• Engineering Memo · External Release

Most email programmes process bounces in batches: after a campaign completes, the bounce report is downloaded, processed against the suppression list, and the updated list is uploaded for the next send. This model introduces a gap — sometimes hours, sometimes days — during which addresses that have already hard-bounced continue to receive messages. At scale, this gap has measurable deliverability consequences and quantifiable list hygiene costs.

Real-time bounce processing at the MTA layer eliminates this gap: when PowerMTA receives a 5XX permanent failure response from a receiving ISP, the address is immediately marked for suppression before the next message in the queue is processed. This note explains the architectural difference between batch and real-time bounce processing, the specific deliverability improvements that real-time processing produces, and how to implement it correctly in PowerMTA and MailWizz environments.

How Batch Bounce Processing Creates Risk

Consider a campaign sending 500,000 messages over 4 hours. Within the first 30 minutes, 3,000 messages return hard bounce responses from Gmail and Yahoo — addresses that no longer exist. With batch bounce processing, these 3,000 addresses remain on the active send list for the remaining 3.5 hours of the campaign. They receive additional delivery attempts (compounding the hard bounce signal at the ISP), and they are not removed from the list before the next campaign triggers.

At ISPs that track repeated sends to invalid addresses — and all major ISPs do — this pattern is a reputation signal. Gmail's sending infrastructure records that this domain consistently attempts delivery to invalid addresses across multiple SMTP sessions. The signal is small per incident, but it accumulates across every campaign that uses the batch processing model. Over months, the accumulated signal of repeated sends to known-invalid addresses contributes to the reputation degradation that manifests as declining inbox placement.

The bounce rate metric itself is also distorted by batch processing. A campaign with 0.8% initial hard bounce rate (above the 0.5% threshold that most email hygiene guidelines flag) may appear as 0.6% in the batch report if some bounces arrived after the report window. The actual bounce rate at the ISP level — which counts all SMTP-level rejection responses regardless of when they are processed — is the one that affects reputation. Batch processing creates a measurement lag that understates the true bounce rate and delays the remediation that the actual rate would trigger.

The MTA-Layer Processing Architecture

MTA-layer bounce processing reads the PowerMTA accounting log in near-real-time — typically with a latency of 10–30 seconds from the SMTP event to the suppression write. The accounting log records every delivery disposition including permanent failures (5XX responses) with the full SMTP response text. A processing daemon reads new accounting log entries, classifies each entry by disposition, and for hard bounce dispositions writes the recipient address to the global suppression database immediately.

In a PowerMTA + MailWizz environment, this real-time processing requires a daemon that reads the PowerMTA accounting log and writes to the MailWizz global blacklist table in near-real-time. The daemon must be capable of: reading the accounting log incrementally (only new entries since last read), identifying hard bounce records by SMTP response code and classification, extracting the recipient email address from the accounting log entry, and performing a parameterised database insert to the MailWizz global blacklist. The daemon should run continuously and be monitored for process health — a stopped daemon reverts the environment to batch processing without visible error.

The second component of MTA-layer bounce processing is PowerMTA's own bounce classification and queue management. When PowerMTA receives a 5XX permanent failure response, it classifies the message as a hard bounce in its internal accounting, moves the message out of the active queue, and does not retry. This prevents the MTA itself from retrying hard-bounced addresses within the current campaign — even before the global suppression database has been updated. The MTA-layer processing thus has two effects: preventing retry within the current campaign send (immediate, handled by PowerMTA) and preventing the address from appearing in future campaign sends (handled by the suppression database update).

The MailWizz Bounce Server Integration

MailWizz uses two bounce processing mechanisms: the accounting log approach described above, and the IMAP-based bounce server that processes Delivery Status Notification (DSN) messages received at the bounce email address. Both mechanisms should be active in a properly configured environment — they complement rather than duplicate each other.

The bounce server (IMAP-based) processes DSN messages that arrive at bounce@yourdomain.com. These messages are generated by receiving mail servers that send a DSN report to the Return-Path address in the original message. For addresses that produce hard bounces, both the SMTP-layer 5XX response (captured by the accounting log) and the DSN message (captured by the bounce server IMAP queue) indicate the permanent failure. Either alone is sufficient for suppression; having both provides redundancy in case one mechanism fails.

The bounce server check interval in MailWizz determines how frequently MailWizz polls the bounce IMAP mailbox for new DSN messages. The default interval is often set to hourly — which is not real-time. Reducing this to every 5–10 minutes approaches real-time behaviour for the IMAP pathway. However, the IMAP pathway is inherently less real-time than the accounting log pathway because it depends on the receiving server generating and sending a DSN message, which introduces variable latency. The accounting log pathway processes the 5XX response at the SMTP layer with 10–30 second latency, regardless of whether a DSN message is generated.

Quantifying the Deliverability Improvement

Moving from batch to real-time bounce processing produces a measurable improvement in hard bounce rate across campaigns, because the suppression gap is eliminated. A programme sending weekly campaigns to a list with 1% monthly address churn (addresses becoming invalid) will, under batch processing, have some portion of those invalid addresses in the active list at send time — how many depends on the batch interval and the rate at which the churn accumulates.

Under real-time processing, invalid addresses are removed from the active list within seconds of the first bounce response. By the next campaign, the entire population of addresses that bounced in previous campaigns is already suppressed. The expected improvement in measured hard bounce rate: 30–60% reduction in campaign hard bounce rates for programmes previously using daily or weekly batch processing. For programmes using per-campaign batch processing (suppressing after each campaign), the improvement is smaller but still meaningful — the within-campaign gap is eliminated.

The reputation improvement is proportional to the bounce rate improvement — fewer sends to invalid addresses means fewer negative signals in ISP reputation databases. For programmes operating near the soft threshold of ISP concern (0.3–0.5% hard bounce rate), the bounce rate reduction from real-time processing can move them below the threshold where ISPs begin applying increased scrutiny to their traffic.

Soft Bounce Handling in Real-Time

While hard bounces (5XX) should be suppressed immediately, soft bounces (4XX) require a different real-time response. The MTA's retry logic handles 4XX responses by queueing the message for retry — this is correct behaviour and should not be interrupted by the bounce processing daemon. Soft bounce counters should be incremented in the accounting log daemon for analytics purposes, but the address should not be suppressed until it reaches the configured soft-to-hard promotion threshold (typically 3 failures across separate campaigns).

The complication: greylisting produces 4XX responses that resolve on retry and should not increment soft bounce counters at all. The accounting log daemon must distinguish greylisting events (4XX followed by 250 OK for the same message) from genuine soft bounces (4XX that persists across the message's retry lifetime). This requires correlating deferral records with eventual delivery or failure records by message ID — a stateful processing requirement that adds complexity to the daemon implementation but is necessary for accurate soft bounce accounting.

In practice, for many MailWizz deployments, soft bounce counting is handled adequately by MailWizz's built-in mechanisms and the accounting log daemon is implemented only for hard bounce real-time suppression. The combination — real-time hard bounce suppression via accounting log + batch soft bounce processing via MailWizz bounce server — eliminates the highest-impact gap (hard bounces) while keeping the implementation simpler than a fully stateful real-time system for both bounce types.

Implementation Considerations and Monitoring

The accounting log daemon is a critical infrastructure component — if it fails silently, the environment reverts to batch processing without any visible alert. Monitoring the daemon requires: process health checks (is the daemon process running?), processing freshness checks (is the daemon reading new log entries within expected latency?), and suppression write success rate checks (are database writes completing without errors?). Alerting on any of these failing should be treated with the same urgency as a delivery rate alert — the daemon failure directly affects list hygiene and deliverability.

The database write destination — the MailWizz global blacklist table — must be accessible to the daemon with write permissions. Database connection pooling is appropriate for high-volume environments where the daemon may be writing hundreds of suppressions per minute during a large campaign. Parameterised queries (not string concatenation) must be used for all database writes to prevent SQL injection risks in the bounce data handling path.

Testing the implementation: after deployment, send a test campaign to a list that includes known-invalid addresses. Verify in the MailWizz global blacklist that the invalid addresses appear within 60 seconds of the bounce being generated. Verify that subsequent campaign sends do not include the suppressed addresses. This end-to-end test validates the full processing chain — accounting log generation, daemon reading, classification, database write, and suppression application — in a controlled environment before relying on it for production list protection.

The Business Case: List Hygiene as Revenue Protection

Hard bounces degrade inbox placement. Inbox placement degradation reduces the percentage of subscribers who see each campaign. Reduced campaign visibility directly reduces campaign revenue. The cost of implementing real-time bounce processing — a daemon implementation and monitoring setup requiring perhaps two days of engineering time — is justified against the revenue impact of even a modest inbox placement improvement over the programme's campaign volume.

The calculation for a programme sending weekly campaigns to 500,000 contacts: if real-time bounce processing reduces the hard bounce rate from 0.6% to 0.3% per campaign (a conservative estimate), the ISP-level improvement in reputation signal volume is 1,500 fewer sends to invalid addresses per campaign, or 78,000 per year. At Gmail's sensitivity to invalid-address sends as a reputation signal, this improvement correlates with a measurable inbox placement benefit — estimated conservatively at 2–3 percentage points for a programme at the threshold of concern.

A 2-point inbox placement improvement on 500,000 contacts means 10,000 more messages reaching the inbox per campaign. At a 1% click-through rate from inbox (conservative for engaged subscribers) and a €60 average order value and 2% conversion from click to purchase, each campaign produces an additional €120 in revenue attributable to the placement improvement. Weekly campaigns over one year: €6,240 per year from a two-day engineering investment. For higher-volume programmes or higher conversion environments, the calculation scales proportionally.

This revenue calculation assumes that the inbox placement improvement is entirely attributable to the bounce rate reduction. In practice, the relationship is not perfectly linear and other factors affect inbox placement. But the directional logic is sound: fewer sends to invalid addresses → fewer negative ISP reputation signals → better reputation standing → better inbox placement → more revenue. Real-time bounce processing is the mechanism that eliminates the most controllable source of invalid-address sends — the gap between when addresses become invalid and when they are suppressed.

FBL Integration: Real-Time Complaint Suppression

The same architectural principle that motivates real-time bounce processing applies to FBL complaint processing. When a recipient marks a message as spam and the ISP sends an ARF complaint to the sender's FBL address, the complainant should be suppressed from future sends within seconds — not at the next batch processing run. Continuing to send to active complainants between FBL receipt and batch processing is a reputation cost that real-time FBL processing eliminates.

Real-time FBL processing requires a listener on the FBL complaint processing endpoint — an email address that receives ARF messages from ISP FBL programs. The listener daemon reads incoming ARF messages, extracts the recipient email address from the original message headers in the ARF body, and writes the address to the global suppression database. Yahoo, Microsoft (SNDS JMRP), and other ISPs that provide FBL services send complaints within minutes to hours of the spam marking event. Real-time processing ensures the suppression takes effect before the complainant receives another message.

Combined real-time bounce and FBL processing creates a hygiene system that closes the two most important suppression gaps: invalid address sends (hard bounce) and active complainant sends (FBL complaint). Both gaps, individually, are reputation-damaging. Together, they are the primary list hygiene mechanism that separates sending programmes with stable High reputation from those that chronically operate at Medium or lower.

Logging and Auditability

The bounce processing daemon should maintain detailed logs of every suppression action: the timestamp, the email address suppressed, the SMTP response code that triggered the suppression, and the source (accounting log or bounce server IMAP). These logs serve several functions: debugging when a valid address appears to have been incorrectly suppressed, audit compliance for GDPR and similar regulations that may require records of suppression actions, and bounce rate analytics that provide the per-ISP and per-campaign breakdown needed for deliverability management decisions.

The suppression log should be retained for at minimum the duration required by applicable data retention regulations. For EU senders under GDPR, the legal basis for retaining email addresses on the suppression list — even after the contact has been suppressed — must be documented: the legitimate interest in not re-sending to addresses that previously produced bounces or complaints. The suppression log, combined with the global blacklist database, constitutes the documentation of this legitimate interest.

Bounce rate analytics drawn from the suppression log provide campaign-level bounce rate data that is more accurate than MailWizz's built-in bounce rate reports, because it captures bounce events from the accounting log in near-real-time rather than from the bounce server IMAP queue with its inherent latency. Weekly bounce rate reports per campaign, per list segment, and per ISP destination — derived from the suppression log — are the data foundation for list quality management decisions: which segments have elevated bounce rates that indicate hygiene problems, which acquisition sources produce higher bounce rates than others, and which ISPs are returning the highest hard bounce volumes.

Real-time bounce processing at the MTA layer is not a luxury enhancement for high-volume senders — it is a basic hygiene mechanism that every professional sending environment should implement. The architectural complexity is modest (a daemon reading accounting logs and writing to a database), the implementation time is days, and the deliverability benefit is immediate and measurable. Programmes that delay implementing it in favour of other deliverability initiatives leave the most controllable source of invalid-address sends operating without correction, compounding reputation damage with every campaign that runs during the gap.

The Suppression List as Infrastructure

A suppression list maintained by real-time bounce and FBL processing is qualitatively different from one maintained by batch processing. It is a live representation of the active, deliverable portion of the contact database — updated within seconds of each bounce or complaint event, rather than in periodic batches. This distinction matters for list quality management decisions: a real-time suppression list allows accurate calculation of deliverable list size at any moment, not just after the latest batch processing run.

Deliverable list size — the count of contacts who are neither suppressed nor pending suppression — is the correct denominator for bounce rate calculations, campaign volume planning, and capacity calculations. Using total list size as the denominator understates bounce rates (because suppressed contacts are not receiving messages) and overstates list value (because it includes addresses that cannot receive messages). Real-time suppression processing keeps the deliverable list size accurate to within seconds, rather than accurate to within the batch interval.

Treating the suppression list as infrastructure — a component that requires monitoring, redundancy, and performance optimisation — reflects the correct operational priority. The suppression list is queried before every message injection, for every campaign, across every sending stream. Its availability and performance affect every send. A suppression list that is unavailable (database down) or performing slowly (query latency above acceptable thresholds) directly affects the ability to send campaigns. Monitoring it with the same rigor applied to the MTA infrastructure — uptime monitoring, query latency alerting, replication lag monitoring — is appropriate for a component with this centrality to the sending pipeline.

The suppression list architecture should also account for growth over time. A list that starts at 50,000 suppressed contacts and grows at 2% per month will have 180,000 contacts within three years. Query performance on a 180,000-row suppression list is different from performance on a 50,000-row list, particularly if suppression checks are done row-by-row rather than in batches. Indexing the suppression table on the email address column (case-insensitive), batch-checking injection lists against the suppression list before each campaign job, and archiving very old suppression records that are no longer relevant are the performance maintenance practices that keep the suppression list responsive as it grows.

Organisations that implement real-time bounce and FBL processing, combined with proper suppression list architecture and monitoring, achieve a level of list hygiene precision that batch-processing environments cannot match. Every campaign they send is to the best available approximation of their current deliverable list — not to a list that was accurate as of the last batch run. This precision compounds over time: cleaner lists produce better engagement signals, better engagement signals produce better reputation, better reputation produces better inbox placement, and better inbox placement produces the campaign performance that justifies the email programme investment. The accounting log daemon and FBL listener are small infrastructure components with outsized influence on this compounding loop.