Pipelines & scenarios · Design & Scenarios

Design a safely retryable pipeline

Hardretriesidempotencyatomic publishdesignreliability

Interview question

How would you design a pipeline that is safe to retry?

How to answer

The short answer

A safely retryable pipeline can be re-run after failure without duplicating or corrupting outputs. is the core design goal.

Design rules

  1. 1 or key your writes (dt=... or business ).
  2. 2Prefer overwrite/merge over blind append.
  3. 3Commit offsets / watermarks only after success.
  4. 4Make side effects safe (emails, webhooks: dedupe tokens).
  5. 5Keep raw inputs durable so you can replay.

Bad vs good

Orchestrator settings

Retries with backoff for transient errors; fail and alert for poison data after N tries; quarantine bad files instead of blocking forever.

Interview tip: Say "retryable = idempotent outputs + late watermark/offset commit + atomic publish." That sentence is the answer.