Retries and failed messages
Railhook retries a delivery on a fixed schedule, the retry ladder, until the endpoint answers 2xx or the ladder runs out. What it gives up on goes to Failed Messages.
Delivery statuses
Section titled “Delivery statuses”| Status | Meaning |
|---|---|
PENDING |
Created, or waiting for its next attempt to be due. |
PROCESSING |
Claimed by a worker for one attempt. If the worker dies, another worker claims the delivery again once the claim times out (WEBHOOK_CLAIM_TIMEOUT_SECONDS, 300 by default). |
SUCCESS |
The endpoint answered 2xx. Nothing more is sent. |
FAILED |
Given up on because nobody can make another attempt help: the URL is refused, or the endpoint cannot receive it. |
DLQ |
The ladder ran out, the endpoint answered with a status another attempt will not change, or the delivery outlived the age cap. Shown in the dashboard as Failed Messages. |
What earns another attempt
Section titled “What earns another attempt”| Result of the attempt | What happens |
|---|---|
2xx |
SUCCESS. |
408, 429, any 5xx |
Retried on the ladder. This is the default set. A subscription or an incoming destination can change it: see Which statuses are worth retrying. |
| A connection error, a timeout, or a host that does not resolve | Retried on the ladder. |
| The transformation failed to apply | Retried on the ladder. The raw payload is never sent instead. |
Any other 4xx, or a 3xx |
DLQ on the first attempt, without the rest of the ladder. Fix the credentials or the URL, then retry from Failed Messages. Redirects are not followed. |
| The endpoint cannot receive it, for example it is not verified, or its URL is refused by the private address check | FAILED without an attempt. |
The retry ladders
Section titled “The retry ladders”Each wait is jittered between 50% and 150% of its value, so retries do not arrive in a burst.
| Direction | Attempts | Waits between attempts | Nominal total |
|---|---|---|---|
| Outgoing (a delivery) | 7 | 1 min, 5 min, 15 min, 1 h, 6 h, 24 h | about 31 h 21 min |
| Incoming (a forward) | 5 | 1 min, 5 min, 15 min, 1 h | about 1 h 21 min |
A subscription or an incoming destination can set its own maxAttempts (1 to 20) and retryDelays in seconds. When there are more attempts than delays, the last delay repeats.
What is not an attempt
Section titled “What is not an attempt”A tenant or per-endpoint rate limit, full concurrency slots, an open circuit breaker, or an ordered delivery waiting for an earlier one stops the request before it is made. That is a deferral: it does not use an attempt or move the ladder, the delivery is just scheduled again.
Timeouts
Section titled “Timeouts”One attempt waits 30 seconds for a response unless the subscription sets timeoutSeconds, which must be between 1 and 60. A timeout is a failed attempt.
Which statuses are worth retrying
Section titled “Which statuses are worth retrying”By default a delivery is retried on 408, 429 and every 5xx, and nothing else. A subscription or an incoming destination can change this with retryableStatuses, a comma-separated list of terms:
| Term | Matches |
|---|---|
429 |
Exactly that status. |
500-599 |
An inclusive range. |
5xx |
Shorthand for 500-599. Also 4xx, 3xx, 2xx, 1xx. |
>=500, >500, <=408, <400 |
Open at one end. |
!501 |
Excluded. Any term can be prefixed with !. |
Exclusions win wherever they are written: 5xx,!501 and !501,5xx mean the same.
408,429,500-599 the default5xx,!501 retry every server error except "not implemented">=500,408,429,409 a receiver that answers 409 while it is still catching up408,429 a receiver whose 5xx means the payload was rejected for goodThe API rejects a list it cannot read and names the bad term. A status the list does not cover is treated like a 4xx: the delivery goes to Failed Messages on the first attempt.
Retry-After
Section titled “Retry-After”A receiver answering 429 or 503 can set Retry-After. Railhook accepts both HTTP forms: seconds or an HTTP date. Three rules bound it:
- Only on
429and503. Only there does it mean “over capacity, wait”. On a500it is often a header a proxy left on. - It can only push an attempt later. A receiver cannot make Railhook retry harder than the subscription owner configured.
- It is clamped, by
WEBHOOK_RETRY_AFTER_MAX_SECONDS(default21600, six hours). Otherwise one misconfigured response could park a delivery for a year.
Six hours is the largest step both default ladders share, and seven attempts at that maximum still fit inside the 96-hour age cap. A longer wait (Hookdeck allows seven days) would be cut short by the cap anyway. Set the variable to 0 to ignore the header.
An honoured Retry-After does not add an attempt. A 429 on the last attempt goes to Failed Messages like any other failure.
The age cap
Section titled “The age cap”A delivery still outstanding 96 hours after it was created is moved to Failed Messages whatever its attempt count. A delivery retried or resent by hand is measured from that moment, so it gets its new attempts. For incoming forwards the cap is 24 hours. The cap catches deliveries that stopped moving; the ladder ends the ones that keep failing.
| Variable | Default | Direction |
|---|---|---|
DELIVERY_ESCALATION_HARD_CAP_HOURS |
96 |
Outgoing |
FORWARD_ESCALATION_HARD_CAP_HOURS |
24 |
Incoming |
The worker refuses to start if a default ladder, at the top of its jitter, would not fit inside its cap.
Failed Messages
Section titled “Failed Messages”Failed Messages lists each abandoned delivery with its endpoint, event type, attempt count and last error. API: GET /api/v1/projects/{projectId}/dlq.
Retrying from Failed Messages puts the same delivery back to PENDING and gives it 3 more attempts. It keeps its delivery id, attempt history and Idempotency-Key, so a receiver that already processed it can detect the duplicate. The first new attempt is sent at once. Retry one delivery with POST /api/v1/projects/{projectId}/dlq/{deliveryId}/retry, or several with POST /api/v1/projects/{projectId}/dlq/retry.
For incoming webhooks the same screen is called Failed Forwards. Retrying one creates a new forward to the same destination, starting from attempt 1.