Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Pub/Sub delivery, ordering and dead letters

Cloud · GCP in Depth

Pub/Sub delivery, ordering and dead letters

Mediumcloud-26
gcppubsubmessagingstreamingreliability

Question

How do acknowledgements, ordering keys and dead-letter topics work in Cloud Pub/Sub?

Solution

Cloud Pub/Sub delivers messages with at-least-once guarantees by default, requiring subscribers to acknowledge each message before an acknowledgment deadline expires to prevent redelivery. When publishers assign ordering keys, Pub/Sub enforces strict per-key delivery ordering within a single region at the expense of lower total partition throughput. If a subscriber repeatedly fails to process a record past a configured retry count, Pub/Sub forwards it to a dead-letter topic, while modern pull subscriptions can enable regional exactly-once delivery.

Message lifecycle and delivery guarantees

Pub/Sub assigns an acknowledgment deadline between 10 and 600 seconds to each message. If the consumer client does not send an acknowledgment before this deadline lapses, the broker assumes the worker died and redelivers the record.

Publisher ---> Pub/Sub Topic ---> Subscription ---> Worker (Ack)
                                        | (Ack deadline expires)
                                        +-----> Worker (Redelivery)

Engineers select between several subscription types depending on consumer architecture:

  • Pull subscriptions allow worker fleets to lease batches of records on demand, giving fine-grained backpressure control.
  • Push subscriptions invoke HTTP webhooks automatically, which works well for serverless endpoints like Cloud Functions.
  • Direct export subscriptions stream data directly into BigQuery tables or Cloud Storage buckets without running compute workers.

Ordering keys and failure isolation

By default, Pub/Sub distributes messages across many internal servers, so overall message order is not preserved. When your pipeline requires sequential processing, you configure ordering keys:

  • All messages sharing the same ordering key are delivered to subscribers in the exact order published, scoped to a single cloud region.
  • Per-key throughput is limited compared to unordered topics because message publishing blocks if earlier messages on that key remain unacknowledged.
  • Regional pull subscriptions offer an exactly-once delivery setting that stops duplicate deliveries caused by transient network retries on acknowledged IDs.
  • Subscriptions configure a dead-letter topic with a maximum delivery attempts threshold between 5 and 100. Poison-pill messages that crash downstream parsers get rerouted to this topic so the main queue keeps moving forward.
PreviousNext