Outcomes & Delivery
Handlers in shove do not write to queues directly. They return one of four Outcome values after processing a message, and the consumer routes the message accordingly. The Outcome is the only mechanism a handler has to influence what happens next.
The four Outcome variants
The Outcome enum has four variants, each mapped to a distinct delivery action:
Ack — the message was handled successfully. The consumer removes it from the queue and moves on. Use this when your handler has done its work: the record was written, the email was sent, the downstream API responded correctly.
Retry — a transient failure occurred. Network blip, downstream service temporarily down, database deadlock — something went wrong that is likely to resolve itself. The consumer routes the message to a hold queue whose index corresponds to the current retry count, waits for the configured delay, then redelivers the message to the main queue. The retry counter increments on each Retry.
Reject — a permanent failure. The message is malformed, violates a business rule, or references data that will never exist. No amount of waiting will fix it. The consumer routes the message to the DLQ if one is configured, or discards it with a warning log if not. The message is never retried.
Defer — the message is not failed, just not yet ready. The consumer routes it to hold_queues[0] (the shortest delay) without incrementing the retry counter. This does not burn the retry budget. See Defer — the special case below.
At-least-once and idempotency
shove guarantees at-least-once delivery by default. Under normal conditions each message is delivered once, but redeliveries can occur after a crash, a network blip, or an unacked timeout. Handlers must be idempotent.
Per-backend deduplication aids exist — RabbitMQ stamped with x-message-id, NATS Nats-Msg-Id, Kafka offset tracking — but these are aids, not guarantees. A fully idempotent handler is the only reliable defence.
Retry math
Hold queue selection is automatic. When a handler returns Retry, the consumer picks:
index = min(retry_count, hold_queues.len() - 1)With hold queues configured at [5s, 30s, 120s] and max_retries = 10:
- 1st retry (
retry_count = 0) → 5s hold queue - 2nd retry (
retry_count = 1) → 30s hold queue - 3rd–10th retry (
retry_count = 2..9) → 120s hold queue (clamped at last index) - 11th attempt exhausted → routed to DLQ (or discarded if no DLQ)
The topic topology defines the hold queues; the consumer options define max_retries. See Retries, Hold Queues & DLQs for a deeper dive.
Reject vs Retry
Choose Reject when the message will never succeed regardless of when it is retried: a malformed payload, a business-rule violation, a foreign-key constraint that will never be satisfied. Retrying those messages wastes resources and burns through the retry budget before the inevitable DLQ routing.
Choose Retry when the failure is transient: a network blip, a downstream service that is temporarily down, a database deadlock. The message is valid and the handler will likely succeed once conditions improve.
When in doubt, ask: "Would this handler succeed if I called it again in five minutes?" If yes, Retry. If no, Reject.
Defer — the special case
Defer exists for messages that are not failed — they are simply not ready yet. Examples:
- A scheduled-for-the-future event that arrived early.
- An order awaiting payment confirmation that has not yet appeared.
- An upstream API that returned 429 (rate limited); the payload is valid, just the timing is wrong.
Defer routes the message to hold_queues[0] (shortest delay) with the same retry count. It does not increment the counter and will never trigger DLQ routing as a result of deferral alone.
One caution: because Defer never increments the retry counter, a handler that always defers will loop the message between the main queue and hold_queues[0] indefinitely. There is no built-in circuit breaker. Ensure your handler has a condition that eventually resolves to Ack, Retry, or Reject.
If no hold queues are configured on the topic, Defer falls back to broker nack-with-requeue and logs a warning.
How a consumer carries out Retry and Defer is its RetryStrategy: Republish, an incremented copy after the tier's delay, or InPlace, the same record delivered again after the tier's delay with nothing published.
InPlace says what is not done, a replacement publish; how the record comes back is the broker's own primitive, in the table below: Kafka waits inside the handler's task and hands the record back, NATS lets JetStream redeliver after a delayed Nak.
One invariant binds every backend: a consumer never writes into an external topology when it settles an outcome, so an external topology implies InPlace everywhere, and each broker has a native primitive for it.
| Backend | In-place primitive | This version |
|---|---|---|
| Apache Kafka | pauses the assignment and waits in the handler's task, the count kept in memory | both strategies; with_retry_strategy on ConsumerOptions::<Kafka> and KafkaConsumerGroupConfig |
| NATS JetStream | naks with the tier's delay, the count from JetStream's num_delivered | InPlace on an external stream, Republish on a shove-owned one; the setter follows |
| AWS SQS | extends the visibility timeout, the path the FIFO consumer already uses | later, additive; declare and every consumer entry point refuse external() |
| RabbitMQ | basic.nack with requeue | later, additive; declare and every consumer entry point refuse external() |
| Redis Streams | leaves the entry pending for idle redelivery | later, additive; declare and every consumer entry point refuse external() |
On Kafka a shove-owned topology runs with Republish by default and may opt into InPlace with with_retry_strategy; an external topology refuses Republish.
Under InPlace nothing is republished: Kafka keeps the count in memory, and NATS reads it from the delivery info, so on an external stream a Defer and an ack_wait expiry consume the retry budget too.
See Bind to an infra-owned topic and Bind to an externally-provisioned stream.
Defer is not supported on sequenced consumers. If a handler returns Defer in that context it is treated as Retry and a warning is logged, because deferral would violate ordering guarantees.
Handler timeouts and deserialization failures
Two failure modes bypass the handler entirely:
Handler timeout. Configured via ConsumerOptions::with_handler_timeout. If the handler future does not complete within the timeout, the consumer aborts it and by default treats the result as Outcome::Retry, so the message is retried according to normal retry math. Which outcome a timeout resolves to is configurable via with_handler_timeout_outcome — see Choosing what a timeout means. Note that a timeout resolved to Defer is subject to the sequenced-consumer restriction above.
Deserialization failure. A message whose bytes cannot be deserialized into T::Message is permanently broken — no amount of retrying will fix a corrupt payload. The consumer treats it as Outcome::Reject without invoking the handler at all. The message goes to the DLQ (or is discarded with a warning) and is never retried. This is by design: poison-pill messages should not burn through the retry budget indefinitely.
For more on handler registration and context, see Handlers & Context.