A supplier invoice can clear upstream capture or Peppol invoice validation and still fail when the integration worker writes a draft into your ERP. ERP maintenance windows, API rate limits, table locks, expired tokens, closed accounting periods, and unmapped vendor records all break imports after you already hold the document.
Repeated retries can consume ERP API capacity and delay other imports. Discarding a failed message can leave an invoice unposted and remove the queue entry operators would use to investigate or replay it.
A dead-letter queue for ERP integrations separates bounded retries from manual diagnosis and operator-controlled redrive. This teardown uses Amazon SQS (a third-party managed queue service) as the concrete reference because its documented policies, retention rules, and redrive limits make the edge cases concrete.
Retry limits, visibility timeouts, and when to dead-letter
In Amazon SQS, a source queue targets a dead-letter queue (DLQ) through a RedrivePolicy with deadLetterTargetArn and maxReceiveCount (valid range 1 to 1,000 per the console guide; default 10 per the SetQueueAttributes API). Once a message’s receive count exceeds maxReceiveCount, SQS moves it to the DLQ.
Two parameters control retries before that handoff:
maxReceiveCount. WithmaxReceiveCount = 1, one unsuccessful receive can exhaust the delivery allowance; setting it too high keeps hitting the ERP with errors that will not clear. On standard queues withmaxReceiveCountabove3, Amazon SQS moves a message to the back of the queue after 3 receives without deletion, so repeated failures do not pinApproximateAgeOfOldestMessageahead of newer imports (Using dead-letter queues).VisibilityTimeout. OnReceiveMessage, SQS hides the message for the visibility timeout (default30seconds; range0to43,200seconds, or12hours, per Amazon SQS visibility timeout). If an ERP import takes 45 seconds against a 30-second timeout, another worker can receive the same message once it becomes visible, while the first write is still in flight, and the receive count goes up. SetVisibilityTimeoutabove worst-case ERP write latency, or extend it withChangeMessageVisibilityheartbeats (extensions never reset the 12-hour limit from first receive).
Route ERP failures by cause rather than sending everything down one retry loop:
| Failure class | Import example | Worker policy | Owner |
|---|---|---|---|
| Transient ERP outage | HTTP 503 maintenance, gateway timeout, row lock | Allow up to maxReceiveCount receives, including the initial delivery, then dead-letter |
Integration ops |
| Slow ERP write | Multi-line invoice exceeds 30 s default |
Extend timeout via ChangeMessageVisibility |
Integration engineering |
| Business validation error | Closed accounting period, missing vendor/G/L map | Log context to an exception table; delete from queue | AP / Finance ops |
| Mapper bug or schema break | Null dereference after upstream field change | Stop at maxReceiveCount; patch worker before redrive |
Integration engineering |
Standard vs. FIFO queues, retention traps, and payload size
Amazon SQS requires creating the DLQ before wiring it, keeping both queues in the same AWS account and Region, and matching queue types—standard to standard, FIFO to FIFO (Configure a dead-letter queue). Restrict RedriveAllowPolicy on finance DLQs to byQueue (up to 10 source queue ARNs) instead of leaving the default allowAll, which lets any source queue in the same account and Region use the DLQ.
Three SQS behaviors create operational traps for ERP teams:
- FIFO ordering breaks on dead-lettering. Within a FIFO queue, messages sharing a
MessageGroupIdprocess sequentially while one is in flight. AWS explicitly warns against attaching a DLQ to a FIFO queue if exact order cannot be broken (Using dead-letter queues). Once a failing invoice moves to the FIFO DLQ, a later message in that vendor’s group—such as a credit note or cancellation—can be processed ahead of it. Redriven messages also interweave with live producer traffic in the destination queue. - Retention expires differently on standard vs. FIFO DLQs.
MessageRetentionPeriodspans60seconds to1,209,600seconds (14days; default4days). On standard queues, expiration stays anchored to the original enqueue timestamp: a message that spends 1 day in the source queue and moves to a DLQ with a 4-day retention expires after 3 days in the DLQ, even thoughApproximateAgeOfOldestMessageon the DLQ counts from arrival. On FIFO queues, the enqueue timestamp resets when the message moves to the DLQ, giving the full 4 days. Always set DLQ retention longer than source retention (up to 14 days), note that loweringMessageRetentionPeriodtakes up to15minutes to propagate and can expire existing messages older than the new period as the change propagates, and archive unresolved failures to durable storage before they age out over a holiday. - Store documents in object storage, not queue bodies. SQS caps messages at
1,048,576bytes (1 MiB) (Amazon SQS message quotas). Keep validated XML or embedded PDFs in durable object storage (for example, Amazon S3—AWS’s synchronous Extended Client Libraries for Java and Python support up to2 GBvia S3 references) and enqueue a compact JSON pointer with document ID, URI, and hash.
Failure context, alerting, and idempotency before redrive
The queue does not know the ERP’s HTTP status, error code, or stack trace; only your worker does. On every failed ERP call, write an attempt row to a database or structured log keyed by document_id, storing the SQS MessageId, ApproximateReceiveCount (from MessageSystemAttributeNames on ReceiveMessage), endpoint, status, raw ERP error, and failure class. Pair the DLQ with an Amazon CloudWatch alarm on ApproximateNumberOfMessagesVisible (Creating alarms for dead-letter queues) so an owner is notified when visible DLQ messages cross the threshold.
Never rely on queue deduplication to prevent duplicate ERP drafts during replay:
- Standard queues provide at-least-once delivery and can deliver a message more than once even inside its visibility timeout.
- FIFO send deduplication applies to
SendMessageretries within a 5-minute interval (Exactly-once processing in Amazon SQS), and moving a message to a FIFO DLQ replaces itsMessageDeduplicationIdwith the original message ID (Configure a DLQ redrive).
Check a durable idempotency key in the import worker before creating a draft, and keep duplicate supplier invoice detection and three-way invoice matching in place downstream.
Illustrative worked example: recovering a failed invoice import
(Illustrative example—not a real company, ERP deployment, or Insonl client.)
An upstream capture service has validated two supplier invoices and written their XML to object storage. Each standard-queue message carries a pointer:
{
"document_id": "DOC-2026-11402",
"vendor_code": "V-1048",
"invoice_number": "INV-2026-8831",
"payload_uri": "s3://ap-validated-docs/2026/11/DOC-2026-11402.xml",
"payload_sha256": "9c2e8a...41f0",
"idempotency_key": "erp-import:V-1048:INV-2026-8831:9c2e8a"
}
The source queue uses VisibilityTimeout = 120 s, MessageRetentionPeriod = 345600 (4 days), and maxReceiveCount = 5, paired with a standard DLQ (MessageRetentionPeriod = 1209600, 14 days, RedriveAllowPolicy = byQueue).
Two failures occur the same evening:
DOC-2026-11402hits an ERP outage. The worker sees five HTTP503responses. The worker logs each attempt toerp_import_attempts(failure_class = "transient_http_503") and leaves the message undeleted. SQS moves the message to the DLQ when its receive count exceeds the configured limit of five, and the CloudWatch alarm onApproximateNumberOfMessagesVisiblethen fires.DOC-2026-11403hits a closed period. Attempt 1 returnsPERIOD_CLOSEDfor2026-10-31. Rather than retrying into the same DLQ, the worker recordsfailure_class = "business_period_closed"in an AP exception table and deletes the queue message; AP re-enqueues the stored pointer once the period opens.
That split matters because Amazon SQS native redrive cannot filter or modify messages (Configure a DLQ redrive). Mixing closed-period errors into the outage DLQ forces either blind re-failure or a custom triage consumer.
After ERP health returns, operators run a four-step recovery:
- Triage and test one import first. If the DLQ holds mixed failure types, separate them before bulk redrive, and verify a single import from its stored document pointer.
- Start a throttled redrive via
StartMessageMoveTask. LeavingDestinationArnblank returns messages to their original source queue; setting it targets a custom queue of the same type (native redrive supports DLQs whose source is an SQS queue only). Set a lowMaxNumberOfMessagesPerSecond—the cap is500msg/s, and AWS recommends starting small and ramping up while watching destination and ERP load. - Monitor or cancel. SQS allows
1active move task per queue, up to100active tasks per account, and36hours max per task. Check status withListMessageMoveTasks(up to10recent tasks) and abort aRUNNINGtask withCancelMessageMoveTask(already-moved messages stay in the destination). Redrive assigns each message a newmessageIDandenqueueTime, resetting retention. - Separate operator IAM permissions. Grant
sqs:StartMessageMoveTask,sqs:CancelMessageMoveTask,sqs:ListMessageMoveTasks,sqs:ReceiveMessage,sqs:DeleteMessage,sqs:GetQueueAttributes, andsqs:ListDeadLetterSourceQueueson the DLQ,sqs:SendMessageon the destination, plus, for SSE-KMS queues,kms:Decrypton any key that encrypted the messages and bothkms:GenerateDataKeyandkms:Decrypton the destination queue’s key. Grant these to an operator role, never the worker.
Where AI fits beside deterministic queue recovery
Keep retry policies, idempotency checks, IAM permissions, and redrive triggers deterministic. Never let an LLM invoke StartMessageMoveTask, delete DLQ entries, or alter invoice payloads.
AI belongs only in read-only triage over the failure log:
- Clustering ERP errors. Grouping raw XML/SOAP or REST failure payloads from
erp_import_attemptsinto distinct incident causes while preserving everydocument_idand raw error code. - Drafting operator triage notes. Matching unfamiliar ERP errors to internal runbooks so an engineer or AP lead can verify prerequisites before authorizing redrive.
Audit your ERP integration recovery path
Insonl is a Ho Chi Minh City engineering studio building ERP systems and operational AI. If failed ERP imports are stalling without bounded retries, losing error context, or risking duplicate drafts on replay, start with our free AI workflow audit.
The audit is free. Follow-on work is optional; Insonl charges only for work you choose afterward.
Sources
- Amazon Web Services, Using dead-letter queues in Amazon SQS: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
- Amazon Web Services, Configure a dead-letter queue using the Amazon SQS console: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-dead-letter-queue.html
- Amazon Web Services, Learn how to configure a dead-letter queue redrive in Amazon SQS: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-dead-letter-queue-redrive.html
- Amazon Web Services, StartMessageMoveTask — Amazon SQS API Reference: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_StartMessageMoveTask.html
- Amazon Web Services, SetQueueAttributes — Amazon SQS API Reference: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html
- Amazon Web Services, ReceiveMessage — Amazon SQS API Reference: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html
- Amazon Web Services, Amazon SQS visibility timeout: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
- Amazon Web Services, Amazon SQS message quotas: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html
- Amazon Web Services, Exactly-once processing in Amazon SQS: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues-exactly-once-processing.html
- Amazon Web Services, Creating alarms for dead-letter queues using Amazon CloudWatch: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/dead-letter-queues-alarms-cloudwatch.html
- Insonl: ERP Systems · Free AI workflow audit · Peppol Invoice Validation Before ERP Import · Duplicate Supplier Invoice Detection · Three-Way Invoice Matching
