Overview
A night job should open one bronze prefix, not the whole company. Least privilege is a JSON statement, not a slogan.
On this page8 sections
The picture
A ShopKart nightly job runs with a key that can open every storage bucket in the company. One mistyped path later, it deletes last year's finance files instead of writing to its own folder.
The job only needed one folder, but it was allowed to touch everything. IAM is the system that decides who may do what, and on which thing.
Words you will see in this lesson
Six words. The rest of the lesson uses only these.
| Word | What it means |
|---|---|
| IAM | Identity and access management. The rulebook for who may do what in your cloud account |
| Identity | Someone or something that can make requests: a person or a program |
| Service account | An identity made for a program. AWS calls it a role, Azure a managed identity |
| Policy | A written rule that allows or denies something |
| Action | One thing a request tries to do, like reading a file |
| Wildcard (*) | A star that means everything. It is the dangerous part of a rule |
Every permission check asks three questions
Whenever a script or a person touches a cloud file, a check runs first. It asks: who is asking, what do they want to do, and to which thing?
Read it left to right. The program asks. The policy is checked. The answer is Allow or Deny.
Who is asking, what do they want to do, and to which thing? IAM answers those three questions.
The same job, two different answers
The service account may read orders/ because a rule says so. It may not read customers/ because no rule says so.
- Read bronze/orders/: rule matches, Allowed
- Read bronze/customers/: no rule, Denied
Cloud IAM works on default deny. If nothing allows an action, it is refused.
Why this shape
Pipelines should only reach their own folder
A pipeline reads raw transactions and writes daily totals. If every job had full access, one bug could wipe the whole lake.
Full access versus scoped access.
| Design | How it works | When something goes wrong |
|---|---|---|
| Full access | Every script can read and write everything | One runaway script can delete production tables |
| Scoped access | Each job can reach only its own folder | The damage stays inside that folder |
Every rule has four parts
- Principal: who the rule is about.
- Action: the verb, such as read a file.
- Resource: the thing it applies to, such as one folder.
- Effect: Allow or Deny.
- Principal (who)
- Action (verb)
- Resource (thing)
- Effect (allow or deny)
AWS, GCP and Azure all use the same four parts. A GCP service account, an AWS role and an Azure managed identity are all just a principal. Learn them once and you can read a policy on any of them.
Every permission rule names who, what action, which thing, and allow or deny.
Walk the boxes
People and programs are different identities
A person signs in with a username, a password and a second check (MFA). A program runs as a service account.
Do not mix them up.
| Identity | Who uses it | How it proves itself | Use it for |
|---|---|---|---|
| Person | Engineers and analysts | Password plus MFA | Looking around in the console |
| Program | Airflow, Spark, cron jobs | A short-lived key from the cloud | Every automated job |
If a pipeline runs under someone's own account, it breaks the day that person changes their password or leaves. Give every scheduled job its own service account.
Actions and roles
An action is one small thing a request can do. On AWS reading a file is s3:GetObject. On Google Cloud it is storage.objects.get.
The same read, with three names.
| Provider | Action | Ready-made role that includes it |
|---|---|---|
| AWS | s3:GetObject | AmazonS3ReadOnlyAccess |
| Google Cloud | storage.objects.get | roles/storage.objectViewer |
| Azure | Microsoft.Storage/blobServices/read | Storage Blob Data Reader |
A role is a bundle of actions, like a job description. You give the identity the role instead of listing every action.
Resources and the dangerous star
A resource is the thing the rule applies to: a bucket, a folder or a table. A star at the end means everything under that path.
The same four parts. Only the scope changes.
| Part | Safe | Dangerous |
|---|---|---|
| Who | This pipeline's own service account | A whole company-wide group |
| Action | storage.objects.get (read only) | * (every action) |
| Resource | gs://lakebench-bronze/orders/* | * (everything) |
| Effect | Allow, for that one folder | Allow, on everything |
In the safe version, the star only covers files inside orders/. A rule with a star in both Action and Resource is a master key to the whole account.
A matching example
Finding the master key in Python
This policy has two rules. One is narrow. One is a master key. The function walks the rules and returns the name of the dangerous one.
Predict: which Sid will it print?
POLICY = {
"Statement": [
{"Sid": "ListBronzePrefix", "Effect": "Allow",
"Action": ["storage.objects.get"], "Resource": "gs://lake/bronze/*"},
{"Sid": "AdminEverywhere", "Effect": "Allow",
"Action": "*", "Resource": "*"},
]
}
def overpermissioned(policy):
for stmt in policy["Statement"]:
action = stmt["Action"]
star = action == "*" or (isinstance(action, list) and "*" in action)
if stmt.get("Effect") == "Allow" and star and stmt["Resource"] == "*":
return stmt["Sid"]
return None
print(overpermissioned(POLICY))AdminEverywhere
An Action can be one string or a list, so the function checks both shapes. It flags the rule only if both the action and the resource are a bare star.
Counting the narrow rules
An auditor may want to know how many rules in a policy are narrow.
def count_least_privilege(policy):
count = 0
for stmt in policy["Statement"]:
action = stmt["Action"]
has_star = action == "*" or (isinstance(action, list) and "*" in action)
resource_star = stmt["Resource"] == "*"
if not has_star and not resource_star:
count += 1
return count
good = count_least_privilege(POLICY)
print("Least-privilege statements:", good)
print("Overpermissioned statements:", len(POLICY["Statement"]) - good)Least-privilege statements: 1 Overpermissioned statements: 1
Teams put a star in while testing and forget it. Treat a star in a real policy as a bug to fix this week. When something goes wrong, the first questions are which identity ran, what action, and on which thing.
Jobs should not hold permanent keys
A permanent key is a long password that never expires. If it leaks, it works for years. The safer way is a temporary key that the cloud hands to the job.
- Give the job a service account (a role on AWS, a managed identity on Azure) and let the cloud hand it a short-lived key.
- Allow only the actions and folders it needs. No star on everything.
- When a tool outside the cloud, like a GitHub build, needs access, use the provider's identity federation (AWS, GCP and Azure each have one). The build proves who it is and gets a temporary key, so no permanent key is stored in the repository.
- GitHub build runs
- It shows its identity
- The cloud checks it
- The cloud gives a temporary key
This way no long-lived key is stored in the code repository.
Watch one check from start to finish
A batch job asks to read bronze orders from cloud storage.
- Batch worker
- Storage request
IAM looks up the rules that apply to the job's service account.
- Service account
- Policy
A rule allows storage.objects.get on orders/*, so the answer is Allow.
- Rule matches
- Allowed
A second request, for customers/*, matches no rule. The answer is Deny.
- No rule
- Denied
Four things to remember:
- Every check looks at who, what action, which thing, and allow or deny.
- Programs run as service accounts with short-lived keys.
- Anything not allowed is denied.
- Do not use a star in both Action and Resource.
Common beginner questions
What if I just use my personal account for everything?
It works until you change your password, leave the company, or hit an MFA prompt at 2 AM. A service account is made for the job: it has no holidays and no password to forget, and you can limit it.
Why are there so many role names?
The providers ship hundreds of ready-made roles so you can be precise. Nobody memorises them. Search for the action you need, such as read objects, and pick the narrowest role that covers it.
Is IAM the same as logging in?
No. Logging in proves who you are. That is called authentication. IAM decides what you may do after that, called authorization. You need both.
What comes next
Next comes Object storage, where the data actually lives. You will see how folders in a bucket work and how IAM rules point at them.
Practice
Run Sample and see which statement your own eye flags as the master key. Then write the function from the exercise: overpermissioned(policy) returns that statement's Sid, and you store its result in a variable called result.
Going further: how a policy decision is actually made
The lesson covered least privilege and policies. The rules for combining them are simple and worth memorising: everything is denied by default, an allow can open an action, and an explicit deny always wins. A small evaluator makes the rules concrete. This is a model of the common cloud logic, not any one provider's engine.
Predict each decision for these policies: one allows reads on the whole raw bucket, another denies everything under the raw/pii folder.
from fnmatch import fnmatch
policies = [
{"effect": "allow", "action": "read", "resource": "raw/*"},
{"effect": "deny", "action": "*", "resource": "raw/pii/*"},
{"effect": "allow", "action": "write", "resource": "curated/sales/*"},
]
def decide(action, resource):
matching = [p for p in policies if fnmatch(action, p["action"]) and fnmatch(resource, p["resource"])]
if any(p["effect"] == "deny" for p in matching):
return "DENY (explicit deny)"
if any(p["effect"] == "allow" for p in matching):
return "ALLOW"
return "DENY (default)"
for action, resource in [("read", "raw/orders/1.csv"), ("read", "raw/pii/users.csv"),
("write", "raw/orders/1.csv"), ("write", "curated/sales/day1")]:
print(f"{action:5} {resource:22} -> {decide(action, resource)}")read raw/orders/1.csv -> ALLOW read raw/pii/users.csv -> DENY (explicit deny) write raw/orders/1.csv -> DENY (default) write curated/sales/day1 -> ALLOW
Reading raw orders is allowed. Reading the pii file matches both an allow (raw/*) and a deny, and the deny wins. Writing to raw matches nothing, so the default applies and it is denied. The write to curated is allowed by the third policy.
This is why least privilege is built from small allows and a few precise denies, rather than one broad allow and hope. A broad allow such as raw/* is only safe because the deny carves out the sensitive part.
Try a new scenario
An analyst role is allowed to read curated/* and is allowed to read raw/*, but a deny blocks everything in raw/pii/*. The analyst complains they cannot open raw/pii/audit.csv, which they need. What are the correct and incorrect fixes?
Work it out, then open the answer
Adding another allow does nothing, because an explicit deny wins over any allow. The correct fix is to change the deny itself, narrowing it so it no longer covers that file, or to grant the access through a separate, reviewed mechanism, after checking whether the analyst should see it at all. Removing the deny for everyone to unblock one person is the incorrect fix.
Interview: what is least privilege?
Give each person or service only the permissions needed for its job and no more, and review them. A smaller set of permissions limits the damage from a mistake or a stolen credential.
Close the lesson and explain it
Without scrolling up, explain the order of default deny, allow and explicit deny, and why adding an allow cannot override a deny.
Practicals · load into the editor
After you read the theory, run these in the pane on the right. They execute in this tab, no cluster.
Finished this lesson?
Mark it as read when you could explain it to someone else.