Cloud · Google Cloud Data Platform

Multi-region streaming resilience with Pub/Sub and dual-region GCS

gcpHardgcpdisaster-recoverypubsubdual-region-gcs

Interview question

How do you architect a disaster-resilient real-time ingestion pipeline across Google Cloud regions?

How to answer

The short answer

Designing high-availability data infrastructure requires ensuring that regional cloud failures do not halt data ingestion, cause data loss, or disrupt downstream executive reporting.

Google Cloud's global software-defined network provides unique building blocks for multi-region resilience.

Pub/Sub global data plane

Unlike traditional messaging brokers that are bound to a single data center, Google Cloud Pub/Sub operates a global data plane:

  • Publishers connect to a single global endpoint (pubsub.googleapis.com).
  • Google routes traffic to the nearest healthy Google point-of-presence worldwide.
  • If an entire Google Cloud region experiences a service interruption, incoming messages automatically route to alternative operational regions without updating publisher DNS or client endpoints.

Dual-region GCS buckets with Turbo Replication

For landing and lakehouse storage, deploy Dual-Region GCS Buckets (such as nam4 pairing Iowa and South Carolina):

  • Dual-region buckets provide active-active read and write capabilities across two distinct geographic zones.
  • Enabling Turbo Replication delivers a service level agreement () guaranteeing that 100% of newly written objects replicate to the paired region within 15 minutes.
  • If Region A fails, applications and analytical pipelines read from Region B using the exact same bucket name and object paths without failover reconfiguration.

BigQuery cross-region disaster recovery

For mission-critical warehouses, configure BigQuery Cross-Region Dataset Replication:

  • BigQuery continuously synchronizes metadata and storage blocks from a primary region (such as us-east4) to a secondary region (such as us-central1).
  • If an outage occurs, administrators execute a single promotion command to switch the replica to primary, delivering an RTO under 15 minutes.