GCP IAM service accounts, Workload Identity, and pipeline security

Service account isolation

A common security mistake is running data pipelines under the default Compute Engine service account, which often possesses broad Editor privileges across the entire project:

• Dedicated service accounts: Every pipeline or service component should have its own service account, such as [email protected].
• Predefined role separation: Assign targeted predefined roles rather than generic project roles. For example, assign roles/storage.objectViewer on source GCS buckets, roles/bigquery.dataEditor on target analytical datasets, and roles/bigquery.jobUser to allow submitting query jobs.

Pipeline Compute (Cloud Run / Dataflow / GKE)
  | Uses Workload Identity (Short-lived OAuth tokens)
  v
Dedicated Service Account (sa-orders-ingest)
  | Granted least-privilege predefined roles
  v
GCS Bucket (objectViewer) + BigQuery Dataset (dataEditor)

Workload Identity Federation and auditing

Storing static JSON service account keys on local developer machines or inside GitHub Actions CI/CD pipelines creates severe security exposure risks.

Workload Identity Federation enables workloads running outside Google Cloud (such as GitHub Actions or on-premise Kubernetes) or inside GKE clusters to impersonate a GCP service account using short-lived tokens, eliminating permanent service account keys entirely.

Every token generation and API invocation is captured in Cloud Audit Logs. This gives security teams complete visibility into which pipeline component accessed specific BigQuery tables or Cloud Storage buckets without managing key rotation schedules.