Every MTA operator has a version of the same story: everything is fine, you go to sleep, you wake up to an alert — or worse, to an angry client — and discover the deferred queue has been growing for six hours because a single ISP started rate-limiting at 3am and nobody noticed until throughput had dropped by 40%. The follow-up conversation is always the same too: "We need better monitoring." The monitoring you need is not complicated to build. What it requires is understanding what the queue health signals actually mean, which ones are worth monitoring, and what response each signal calls for.
This guide is the queue management reference I wish existed when I was first running production MTAs. Not the theoretical description of what queues are, but the operational playbook for what you do when they are not behaving correctly.
Reading Queue Health Before It Becomes a Problem
The three numbers that tell you queue health at a glance — for both Postfix and PowerMTA:
# Postfix — quick queue health check:
postqueue -p | tail -1
# Output: "-- N Kbytes in M Requests."
# N = size of active+deferred queues in kilobytes
# M = number of messages in active+deferred queues
# Baseline: know your normal M value; deviations indicate problems
# Breakdown by queue:
for queue in incoming active deferred bounce; do
count=$(find /var/spool/postfix/$queue -type f 2>/dev/null | wc -l)
echo "$queue: $count"
done
# Active should be close to qmgr_message_active_limit or naturally cycling
# Deferred should be low (< 10% of daily send volume) under normal conditions
# Bounce should be near zero; rising bounce count = list quality problem
# PowerMTA — queue status:
pmta show queue
# Shows: message count, volume, rate per minute, per VMTA and destination
# Per-destination breakdown in PowerMTA:
pmta show queue --status-file /var/log/pmta/queue-status.csv
# Or view in web interface: http://your-pmta:8080/
# What "normal" looks like vs what "problem" looks like:
# NORMAL: deferred < 5% of daily volume, decreasing over time
# CONCERN: deferred 10-20% of daily volume, stable (not increasing or decreasing)
# PROBLEM: deferred > 20% of daily volume, increasing
# EMERGENCY: deferred > 50% of daily volume, increasing rapidly
The key insight about the deferred queue: a message in the deferred queue has not failed. It was temporarily declined by the receiving server and is waiting for a retry. The deferred queue becomes a failure when the retry timer expires and the message bounces — which for most MTAs is configured at 4-5 days. A deferred queue that grows at 3am and resolves itself by 7am when ISP rate limits reset is operationally normal, albeit potentially visible as a delay to time-sensitive email. A deferred queue that grows and does not resolve — because the ISP has implemented a hard block, or the IP has been listed in a blocklist — will convert to bounces after the retry expiry and represents real delivery failure requiring intervention.
The Deferred Queue: What Each Deferral Type Tells You
Not all deferrals are the same problem. The specific SMTP response code in the deferral record tells you what is happening at the receiving end:
# Analysing deferred queue reasons in Postfix:
postqueue -p | grep -A2 "^[0-9A-F]" | grep "4\." | sort | uniq -c | sort -rn | head -20
# Or for more detail, parse the defer log:
grep "status=deferred" /var/log/mail.log | grep -oP "dsn=\K[^,]+" | sort | uniq -c | sort -rn
# What each code means:
# 4.7.650 - Microsoft rate limiting (too much volume for current reputation)
# 4.2.1 - Yahoo "too many connections" (connection concurrency too high)
# 4.1.1 - Receiving server does not know this user (soft bounce first occurrence)
# 4.4.1 - Connection refused (ISP closing connections)
# 4.4.2 - Connection timed out (server not responding)
# 4.7.1 - Delivery not authorized (policy-based, may be authentication failure)
# 4.3.2 - System not accepting network messages (ISP maintenance or overload)
# In PowerMTA accounting log:
awk -F',' '$1=="t" {print $9}' /var/log/pmta/acct.csv | sort | uniq -c | sort -rn | head -20
# Column 9 in PowerMTA CSV is the SMTP response string for transient failures
The Microsoft 4.7.650 code is the most common ISP-specific deferral I see in production. It means Microsoft has placed the sending IP in temporary rate limiting — not a blocklist listing, not a permanent rejection, but a throttle that limits throughput to what Microsoft considers appropriate for the IP's current reputation. The standard response is to reduce concurrent connections to Microsoft domains (as covered in the Postfix and PowerMTA tuning guides on this site) and wait — typically 60-90 minutes — for the rate limiter to reset. Do not attempt to push harder through a 4.7.650 rate limit; Microsoft's system will extend the limiting period in response to continued high-rate delivery attempts.
The 4.4.2 connection timeout deferral looks similar to the 4.4.1 connection refused but has different causes. Connection refused (4.4.1) means the ISP is actively rejecting your connections — the TCP handshake is completing and then being rejected. Connection timeout (4.4.2) means the TCP handshake is not completing — either the receiving server is unavailable, the firewall is silently dropping connections, or DNS is resolving to an incorrect or unreachable IP. When you see persistent 4.4.2 deferrals to a specific ISP, check whether the MX records for that ISP resolve correctly and whether you can establish a manual TCP connection to port 25 on the MX servers.
ISP Rate-Limit Jams: Identification and Response
An ISP rate-limit jam is when a specific ISP begins deferring a significant fraction of your email simultaneously, causing the deferred queue to grow rapidly with messages destined for that ISP. This is the most common cause of sudden deferred queue growth, and it requires a specific response sequence:
1. Identify which ISP and how many messages:
# For Postfix — find which domains are accumulating in the deferred queue:
postqueue -p | grep "@" | grep -oP "@\K[^>]+" | awk -F'.' 'NF>=2{print $(NF-1)"."$NF}' | sort | uniq -c | sort -rn | head -10
# For PowerMTA — per-destination queue breakdown:
pmta show queue | grep -E "hotmail|outlook|yahoo|gmail" | sort -k2 -rn
# Expected output: the ISP accumulating messages stands out
# e.g., "hotmail.com: 47,283 messages" where normal is < 5,000
2. Determine the cause from the SMTP response: If it is 4.7.650 at Microsoft, it is rate limiting — the response is to reduce concurrency and wait. If it is 4.2.1 at Yahoo, it is connection overload — reduce connections. If it is any 5xx code for a significant fraction of messages, you have shifted from a deferral (temporary) to a block (permanent) and the response is different: investigation into SNDS/Postmaster Tools for the cause.
3. Implement the immediate response: For rate limiting, the response is to temporarily pause or reduce sending to the affected ISP to let the rate limiter reset. In PowerMTA:
# PowerMTA: temporarily reduce concurrency to Microsoft pmta set smtp hotmail.com max-smtp-out 3 # Reduce from normal 12 to 3 while the rate limiter resets # After 90 minutes, restore: # pmta set smtp hotmail.com max-smtp-out 12 # PowerMTA: flush messages to a specific domain after resolving block: pmta flush domain hotmail.com # This retries all deferred messages to hotmail.com immediately # Postfix: force retry of deferred messages to specific domain: postqueue -s hotmail.com # This tells the queue manager to retry immediately for hotmail.com # Postfix: adjust Microsoft connection count temporarily: # (Requires transport map and per-transport configuration — see Postfix tuning guide) # Easiest quick fix: postfix stop; edit main.cf with lower smtp_destination_concurrency_limit; # postfix start; restore after 90 minutes
Queue Priority: Ensuring Transactional Email Clears First
When the queue backs up — for any reason — transactional email (order confirmations, password resets, account alerts) must clear before promotional email. A customer who placed an order is expecting their confirmation; a subscriber who hasn't opened the last 12 newsletters can wait an extra 4 hours for the next one. The priority architecture that makes this work:
In Postfix: Use separate transport maps for transactional and promotional email (as described in the Postfix tuning guide), with the transactional transport having higher priority. Postfix does not have a native queue priority setting, but you can achieve functional priority by giving the transactional transport more concurrency — it will be served preferentially by the queue manager when both queues have messages.
In PowerMTA: Use separate VMTAs for transactional and promotional email. Set higher max-smtp-out on the transactional VMTA, and configure separate source IPs with separate warmup state. PowerMTA's queue manager prioritises based on pool resource availability, so the VMTA with more available SMTP processes gets serviced first. For explicit message-level priority:
# PowerMTA message-level priority (set via X-Priority header or metadata): # In the VMTA configuration, define priority routing: <virtual-mta transactional-priority> smtp-source-host 198.51.100.10 trans1.mail.brand.com # Higher concurrency = higher effective priority max-smtp-out 100 # Optional: use pmta's priority field in accounting # Set message priority 1 (highest) for transactional, 5 for marketing # Priority can be set via X-Priority header or via MailWizz campaign settings </virtual-mta> <virtual-mta marketing-normal> smtp-source-host 198.51.100.11 mkt1.mail.brand.com max-smtp-out 30 # Lower concurrency = lower effective priority when competing with transactional </virtual-mta>
Bulk Queue Operations: Deleting, Requeuing, and Rerouting
Situations that require bulk queue operations: (1) A campaign was sent with an error (wrong content, wrong list) and needs to be cancelled before delivery. (2) A domain is temporarily unreachable and the deferred queue for that domain needs to be flushed when it comes back online. (3) A sending IP is being retired and all queued messages need to be rerouted to a replacement IP. (4) An authentication misconfiguration was discovered after a large send and queued messages need to be held while the fix is applied.
# Postfix: delete all deferred messages to a specific domain:
postqueue -p | grep "@target-domain.com" | grep -oP "[0-9A-F]{10,}" | head -10000 | while read id; do
postsuper -d $id
done
# Warning: this permanently deletes — use with caution
# Test with a small head -10 count first
# Postfix: hold all messages in the deferred queue (emergency stop):
postsuper -h ALL deferred
# Messages move to "hold" queue — delivery stops
# To release:
# postsuper -H ALL
# Postfix: requeue messages from hold back to active:
postsuper -r ALL hold
# This retries all held messages immediately
# PowerMTA: delete messages for a specific destination:
pmta delete --dest hotmail.com
# Removes all queued messages to hotmail.com — use in genuine emergencies
# PowerMTA: view specific message by queue ID:
pmta show message --msgid MESSAGE_ID
# PowerMTA: inject a new message (for testing queue processing):
pmta inject --from sender@domain.com --rcpt rcpt@test.com < /tmp/test.eml
The most operationally dangerous queue operation is the bulk delete. Once messages are deleted from the queue, they are gone — there is no undo, no recovery. Before performing any bulk delete, verify with the sending platform (MailWizz, or whoever injected the messages) that the send cannot simply be cancelled at the application layer — which is always preferable to deleting at the MTA queue level. If the application layer cancel is not possible (because the messages have already been injected and the application has no record of the queue IDs), then bulk delete is the correct tool, but the scope should be verified carefully before execution.
Automated Queue Monitoring That Pages When It Matters
#!/usr/bin/env bash
# /usr/local/bin/check-queue-health.sh
# Run every 5 minutes via cron; send alert to monitoring system
ALERT_EMAIL="ops@yourcompany.com"
DEFERRED_WARNING=5000 # Alert if deferred queue exceeds this
DEFERRED_CRITICAL=20000 # Critical alert at this threshold
BOUNCE_WARNING=500 # Alert if bounce count exceeds this per hour
# Get current queue counts
DEFERRED=$(find /var/spool/postfix/deferred -type f 2>/dev/null | wc -l)
BOUNCE=$(find /var/spool/postfix/bounce -type f 2>/dev/null | wc -l)
# Determine alert level
ALERT_LEVEL="OK"
ALERT_MSG=""
if [ $DEFERRED -gt $DEFERRED_CRITICAL ]; then
ALERT_LEVEL="CRITICAL"
ALERT_MSG="Deferred queue CRITICAL: $DEFERRED messages"
elif [ $DEFERRED -gt $DEFERRED_WARNING ]; then
ALERT_LEVEL="WARNING"
ALERT_MSG="Deferred queue WARNING: $DEFERRED messages"
fi
if [ $BOUNCE -gt $BOUNCE_WARNING ]; then
ALERT_LEVEL="WARNING"
ALERT_MSG="$ALERT_MSG | Bounce queue WARNING: $BOUNCE messages"
fi
# Send alert if not OK
if [ "$ALERT_LEVEL" != "OK" ]; then
# Get top deferring destinations for context
TOP_DEFERRED=$(postqueue -p | grep "@" | grep -oP "@\K[^>]+" | awk -F'.' 'NF>=2{print $(NF-1)"."$NF}' | sort | uniq -c | sort -rn | head -5)
echo "Queue Health Alert: $ALERT_LEVEL
Time: $(date)
Deferred: $DEFERRED
Bounce: $BOUNCE
Top deferred destinations:
$TOP_DEFERRED
Action: Review /var/log/mail.log for deferral reasons" | mail -s "[$ALERT_LEVEL] MTA Queue Alert: $ALERT_MSG" $ALERT_EMAIL
fi
# Log for trending (useful for graphing in Grafana/Prometheus):
echo "$(date +%s),deferred,$DEFERRED" >> /var/log/pmta-metrics/queue-counts.csv
echo "$(date +%s),bounce,$BOUNCE" >> /var/log/pmta-metrics/queue-counts.csv
PowerMTA Queue Tools: pmta Command Reference
# Essential pmta queue management commands: # Show overall queue status: pmta show queue # Show queue for specific VMTA: pmta show queue --vmta vmta-name # Show queue for specific destination: pmta show queue --dest gmail.com # Flush (retry immediately) deferred messages to specific domain: pmta flush domain gmail.com # Flush all deferred messages (use sparingly — causes connection spike): pmta flush all # Pause delivery to a destination (useful during ISP maintenance windows): pmta set smtp gmail.com pause # Resume delivery: pmta set smtp gmail.com resume # Show current SMTP connections: pmta show connection # Show per-VMTA throughput rates: pmta show accounting # View message by ID: pmta show message --msgid ID # The PowerMTA web management interface (if enabled): # http://your-pmta-server:8080/ # Provides real-time queue visualisation and per-VMTA throughput graphs # More useful than command-line for getting overview at a glance
Postfix Queue Operations: The Commands You Actually Need
# Queue inspection: postqueue -p # Full queue listing postqueue -p | wc -l # Total message count (subtract 1 for header line) mailq # Alias for postqueue -p postqueue -p | tail -1 # Just the summary line: N Kbytes in M Requests # Queue manipulation: postsuper -d MESSAGE_ID # Delete specific message postsuper -d ALL deferred # Delete all deferred messages (DESTRUCTIVE) postsuper -h MESSAGE_ID # Hold specific message postsuper -h ALL deferred # Hold all deferred messages postsuper -H MESSAGE_ID # Release held message postsuper -H ALL # Release all held messages postsuper -r MESSAGE_ID # Requeue specific message (move to incoming) postsuper -r ALL deferred # Requeue all deferred (force immediate retry) # Forced retry: postqueue -f # Flush deferred queue (retry all immediately) postqueue -s domain.com # Retry deferred messages to specific domain # Reading a specific message: postcat -q MESSAGE_ID # Show message content from queue # Queue statistics: qshape deferred # Show deferred queue by domain and age buckets qshape active # Show active queue distribution # The qshape output is extremely useful for identifying where messages # are ageing in the deferred queue: # qshape deferred | head -20 # Shows: domain, total count, and age distribution (5min, 20min, 1hr, 4hr, 12hr, 24hr, 48hr+) # Any domain with significant 24hr+ entries needs investigation
Queue management is the daily operational reality of running email infrastructure at scale. The difference between operators who handle queue events smoothly and those who scramble is not technical knowledge — it is the combination of monitoring that detects problems early (before the deferred queue grows to the point where it affects customer-facing delivery), a clear mental model of what each queue signal means and what response it requires, and the practiced confidence with queue manipulation commands that lets you take appropriate action quickly when problems do arise. None of these skills come from reading documentation alone; they come from operating production infrastructure and accumulating the specific knowledge that documentation does not capture — like knowing that a Microsoft 4.7.650 rate limit at 3am will self-resolve by 5am if you reduce concurrency and wait, or that Postfix's postqueue -s domain.com releases deferred messages without the connection spike of postqueue -f on the entire deferred queue. Build the monitoring, practice the commands in non-crisis moments, and the 2am crisis becomes a 5-minute resolution rather than a 2-hour scramble.