Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Design a GDPR/DPDP-compliant user data platform

Pipelines & scenarios · System Design Questions

Design a GDPR/DPDP-compliant user data platform

Hardpipelines-38
scenariogdprprivacydeletionpii

Question

Design a data platform that must support "delete my data" requests within 30 days.

Solution

A "delete my data" promise is only keepable if you know where every copy of personal data lives, and can find it by a user id. So the design starts with an inventory and with how PII is stored, and the deletion job comes last.

Know where PII is

Create an inventory of every source, table and column that holds personal data, and tag the columns (email, phone, address, device id). The catalog should show lineage, so you can see which derived tables, extracts and features copy those columns. Without lineage, you cannot honestly say you deleted everything.

Keep PII in one place

The best structural choice is to keep personal fields in a few dedicated tables, keyed by a surrogate user_key, and keep everything else (orders, events, metrics) keyed only by that surrogate id. Analytics runs on the second group. Deleting a user then means deleting rows in the PII tables, and the rest becomes anonymous, because nothing links it back to a person.

dim_user_pii (user_key, name, email, phone)   <- delete here
fact_orders  (order_id, user_key, amount)     <- stays, now unlinkable

Tokenize or hash identifiers at ingestion where you can, so raw email addresses do not spread through the lake.

The deletion workflow

  • A request intake (a ticket or API) creates a record with a deadline (30 days is the limit asked here).
  • A scheduled job resolves the person to their ids across systems, and runs deletes in the warehouse, lake tables, feature stores, search indexes and caches.
  • On table formats such as Delta or Iceberg, DELETE only marks rows removed. Files are physically purged after compaction and VACUUM or snapshot expiry, so schedule those within the deadline.
  • Backups: you usually cannot edit them. Options are to document a retention period shorter than the legal limit, or to use crypto-shredding.

Crypto-shredding

Encrypt each user's PII with a per-user key, and keep keys in a key management service. To delete the user, destroy their key, and every copy of the encrypted data (including backups and logs) becomes unreadable. It is powerful, but adds complexity, and you must make sure no plain-text copies exist.

Proof

Keep an audit trail: who asked, what was deleted where, and when. Run a verification query afterwards that searches for the user's identifiers and expects zero results. Report the status to the privacy team.

Also

Some data must be kept (invoices for tax law), so the workflow needs rules for exceptions. Say that you would consult the privacy and legal teams on those, since the engineering design follows their decisions.

PreviousNext