Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Dataplex and governance on GCP

Cloud · GCP in Depth

Dataplex and governance on GCP

Mediumcloud-32
gcpdataplexgovernancedata-catalogmetadata

Question

What is Dataplex used for?

Solution

Google Cloud Dataplex provides centralized governance, discovery, and quality management across distributed data lakes in Cloud Storage and warehouses in BigQuery. It organizes analytical assets into logical lakes and zones without copying physical data files, automating metadata discovery through Dataplex Universal Catalog. Teams use Dataplex to run scheduled data quality scans, track automated data lineage, generate column profiling metrics, and enforce column-level security policies.

Logical domain organization

Dataplex decouples storage location from governance administration, grouping disparate storage buckets and BigQuery datasets into coherent business domains.

Dataplex Lake (e.g., Marketing Domain)                               ├── Raw Zone      ---> GCS Buckets (Bronze files, JSON/CSV)      └── Curated Zone  ---> BigQuery Datasets (Silver/Gold tables)             |                                                       Central Governance (Catalog, Lineage, Policy Tags, Scans)    

This logical structure provides several governance capabilities:

  • Lakes and zones establish administrative boundaries. Raw zones map to landing buckets with relaxed schemas, while curated zones encompass clean BigQuery tables with strict quality rules.
  • Universal Catalog automatically inventories assets, crawls schemas, and exposes search capabilities across both object storage buckets and analytical warehouse tables.
  • Data lineage tracks transformations automatically from BigQuery SQL executions, Dataflow pipelines, and Composer workflows, letting engineers trace downstream impact before making table changes.
  • Automated data profiling computes column statistics, null rates, and value distributions without writing custom SQL verification scripts.

Security policies and BigQuery integration

Dataplex works alongside foundational Google Cloud security controls to protect sensitive fields:

  • Automated data quality scans evaluate declarative rules directly against target tables, producing pass and fail scorecards and triggering alerts on anomalies.
  • Policy tags managed in Dataplex integrate with BigQuery column-level access controls, masking personally identifiable information like social security numbers from unauthorized analysts.
  • Dataplex does not replace standard IAM permissions; it complements BigQuery IAM by centralizing policy definitions and auditing compliance across multiple Google Cloud projects.
PreviousNext