Lesson 1 of 4 in this module

1/14 in track

What is cloud computing?

python12 min

Overview

Rent compute, storage, and network from a provider instead of buying servers. No cloud account lives in this tab.

On this page8 sections

The idea

A shop that runs on one laptop

Picture a small online shop in Mumbai. Every order lands in a spreadsheet on the owner's laptop.

It works for a month. Then the laptop falls off the desk, or the battery dies in the middle of an export, or the disk fills up.

  1. Customer orders
  2. Order saved on the laptop
  3. Laptop breaks, orders stop

All three problems have one cause. The whole business depends on one machine. If that machine stops, everything stops.

One machine in one room is a single point of failure. Cloud is the way out of it.

Words you will see in this lesson

Six words. The rest of the lesson uses only these.

WordWhat it means
CloudComputers you rent from a company and use over the internet, instead of buying them
ProviderThe company that owns those computers. AWS, Google Cloud and Azure are the big three
Virtual machine (VM)A computer you rent by the hour. It lives in the provider's building, not in your office
BucketA folder in the cloud where you keep files. The service behind it is called object storage
RegionA part of the world where the provider has buildings, like Mumbai
PipelineA set of steps that moves data from where it starts to where people use it

Renting instead of buying

Cloud computing means you stop buying computers and start renting them.

You pay for the hours you use or the gigabytes you store. When the job ends, you give the machine back and the bill stops.

The same computing, two ways to pay for it.

Buy your ownRent from a cloud provider
Where the machines liveYour office or a server roomThe provider's big buildings
How you payA large amount, years before you need itBy the second or the gigabyte
When the job is overThe machine sits there, switched onYou delete it and billing stops

Amazon Web Services (AWS), Google Cloud Platform (GCP) and Microsoft Azure are the three biggest providers.

Why this exists

Nobody can guess next year's traffic

Before the cloud, a company had to decide how many servers to buy years ahead.

Buy too few and the site crashes on the busiest day. Buy too many and they sit idle for the rest of the year.

The same shop on three different days.

DayOwn serversRented machines
Normal day, 10,000 ordersOne server is enough1 small machine
Diwali sale, 100,000 ordersThe server runs out of memory and crashes10 machines for 2 hours
After the saleExtra servers sit idle but still cost moneyThe 10 machines are deleted, cost drops to zero

Renting lets you pay for 10 machines for 2 hours, not 10 machines for a year.

Storage and compute can be rented separately

On your own server, the disk and the processor sit in the same box. If you need more disk, you buy a whole new box, even if the processor sits idle.

In the cloud they are separate products. Files sit in a bucket at a low price per gigabyte. You rent a machine only when you need to work on them.

  1. Own server: more disk means a whole new box
  2. Cloud: files in a bucket, machines rented only when needed

A dead disk does not lose your files

A hard disk on your desk can die, and the data on it dies with it.

Cloud storage keeps several copies of every file on different machines, and usually in different buildings. One broken disk, or one building losing power, does not touch your files.

  1. You upload a file
  2. Provider stores several copies
  1. A disk fails, the file is still there

The provider also has staff in the building who replace broken hardware. You never see it happen.

Picture this

Like hiring a machine for a weekend job

Think of renting an excavator for a weekend of garden work. You pay by the hour, and on Sunday evening you return it.

A cloud provider is that rental shop, but for computing power.

A room with rows of black server racks and cabling on a raised tile floor
A real server room. Each rack holds dozens of computers stacked on top of each other. Renting a virtual machine means renting a slice of one of these.Source: Carl Lender via Wikimedia Commons · CC BY 2.0

Nothing here is imaginary. These are real computers in real buildings. The difference is how you get one: you send a request and a machine is ready in a couple of minutes.

Regions and zones

Aerial view at sunset of a large grey data center building with cooling equipment along one side
One data center building seen from above. A cloud region is a group of buildings like this in the same part of the world.Source: Chad Davis via Wikimedia Commons · CC BY 2.0

Providers group their buildings into regions, such as Mumbai. Inside a region, each separate building with its own power is called an availability zone.

The same idea, named differently by each provider.

IdeaAWS nameGoogle Cloud nameWhat it is
Region (Mumbai)ap-south-1asia-south1A group of buildings near Mumbai
Zoneap-south-1aasia-south1-aOne building with its own power
Region (US east)us-east-1us-east4A group of buildings in Virginia

Pick a region close to your users and your data. A request from Mumbai to Virginia travels across the ocean, so it is slower.

Everything is one of three things

A cloud catalog has hundreds of products. Almost every one of them is compute, storage or network.

The three things you rent
Compute: the CPU and memory that run your codeStorage: where files and tables live when nothing is runningNetwork: the wires and rules that let the other two talk

Every cloud account is built from these. A pipeline needs all three.

The same three jobs. Only the way you pay changed.

You used to buyYou now rentWhat it gives a pipeline
A server in a closetA virtual machineThe job runs at 2 AM, even if every laptop is switched off
Hard disks and a file serverA bucketRaw files survive when one machine dies
Cables and a firewall boxA private networkThe database is not exposed to the whole internet
Watch outRenting means you can forget to switch it off

A virtual machine keeps billing while it sits idle. The classic beginner bill is a machine started on Friday for a quick test and left running all weekend. Always delete what you do not need.

Two ways beginners get a surprise bill

Both show up on the bill, not on the screen.

TrapHow it happensHow to avoid it
Forgotten machineYou start a big machine on Friday and forget itDelete it when you finish, or set a timer that shuts it down
Moving data between regionsA machine in Mumbai reads a bucket in Virginia, and every gigabyte is chargedKeep the machine and the bucket in the same region

Uploading data to the cloud is usually free. Taking data out, or moving it between regions, costs money.

A real night in a small shop

ShopKart is an online store in Bengaluru. Orders arrive all day.

  1. Daytime: each order is saved as a file in a bucket
  1. 2:00 AM: a timer starts a rented machine
  1. 2:35 AM: the machine cleans the data and builds the report
  1. 2:40 AM: the machine is deleted, the report is ready

During the day the order files pile up in a bucket. At 2 AM a timer starts a machine, which cleans the files and adds up the sales.

The totals go into a table. The machine then shuts down. The analyst opens the report at 8:30 AM and sees fresh numbers.

A small example

The three things in Python

This tab cannot create a real cloud account. The Python editor runs in this tab and uses lists and dicts to stand in for the real thing.

Predict first: how many items will this print?

PythonName the three things
PILLARS = ["compute", "storage", "network"]
print("A cloud account rents:")
for name in PILLARS:
    print("-", name)
print("count", len(PILLARS))
A cloud account rents:
- compute
- storage
- network
count 3

Three lines, one per item, then the count. Every cloud product you will meet is one of these or a mix of them.

Mapping the nightly job

Take the ShopKart job and write down what each of the three things does for it.

PythonThe shop's nightly job
NIGHTLY_JOB = {
    "compute": "a small VM that runs the report script for about 2 hours",
    "storage": "a bucket that holds the day's order files",
    "network": "a private route from the VM to the reporting database",
}
for pillar, need in NIGHTLY_JOB.items():
    print(pillar, "->", need)
compute -> a small VM that runs the report script for about 2 hours
storage -> a bucket that holds the day's order files
network -> a private route from the VM to the reporting database

The machine only costs money for the 2 hours it runs. The bucket costs a small amount all month. The private route costs almost nothing.

RememberWhere this tab stops

The exercises here print Python lists and dicts. They never create a machine, a bucket or a bill. A real free-tier project comes later in the track.

Estimating the cost before you start

Engineers work out the bill before they launch anything: hours multiplied by the hourly price, plus gigabytes multiplied by the monthly price.

Predict: what will 60 hours of machine time and 500 gigabytes of storage cost?

PythonCalculate the monthly cost
def estimate_cost(hours, storage_gb):
    vm_rate = 0.04
    disk_rate = 0.02
    return round((hours * vm_rate) + (storage_gb * disk_rate), 2)

print("Nightly batch cost estimate:")
print("Monthly total: $", estimate_cost(60, 500))
Nightly batch cost estimate:
Monthly total: $ 12.4

The machine part is only $2.40 of the $12.40. The storage is the bigger share, because it is paid for all month. The prices here are made up to keep the sums easy.

Watch it happen, step by step

ShopKart nightly run
  1. During the day, each order is saved as a file in a bucket.

    1. Customer checkout
    2. Bucket
  2. The files just sit there. Storage is cheap and needs no machine.

    1. Bucket
    2. No machine running
  3. At 2:00 AM a timer starts a rented machine in the same region.

    1. Timer
    2. Machine in Mumbai
  4. The machine builds the report table and is deleted before morning.

    1. Report table
    2. Machine deleted

Four things to remember:

  1. Cloud means renting computers over the internet and paying for what you use.
  2. Files and machines are rented separately, so you can grow one without the other.
  3. Keep your machine and your bucket in the same region to avoid transfer charges.
  4. Delete machines you are not using, or they keep billing.

Common beginner questions

Is the cloud just someone else's computer?

Mostly, yes. What it adds is that you create and delete machines by sending a request, you pay only for what you use, and your files are copied to more than one building.

Do I need to learn all three providers?

No. Learn one reasonably well and know what the other two call the same things. The next lesson puts AWS, Azure and Google Cloud side by side.

Can I practise without a credit card?

In this tab, yes. Nothing here costs anything. Real signup rules differ by provider, country and time, and some ask for a card even for the free tier, so read the current terms before you sign up.

What comes next

Next, Cloud services overview opens the catalog: the five kinds of product every provider sells, and what AWS, Azure and Google Cloud each call them.

Practice

Run Sample to see the three things printed. Then do the exercise: a team lists four infrastructure needs, and you decide for each one if it is compute, storage or network. Build a dict called result and print it.

Going further: when does paying per use win?

The cloud's main idea is that you rent capacity by the hour instead of buying machines. That wins when your load is spiky and loses when it is flat and constant.

You can see it with arithmetic. The prices below are made-up round numbers to show the method; use your provider's real price list for real decisions.

Assume a server costs 2,000 per month to own and run, and the same capacity costs 4 per hour in the cloud. A job runs 3 hours a day in one case and all day in the other. Predict which option is cheaper in each case.

PythonRent by the hour or own the machine
OWN_PER_MONTH = 2000.0        # illustrative number
CLOUD_PER_HOUR = 4.0          # illustrative number

for label, hours_per_day in [("job runs 3 hours a day", 3), ("job runs 24 hours a day", 24)]:
    cloud = CLOUD_PER_HOUR * hours_per_day * 30
    cheaper = "cloud" if cloud < OWN_PER_MONTH else "own"
    print(f"{label:25} cloud = {cloud:7.0f}   own = {OWN_PER_MONTH:7.0f}   cheaper: {cheaper}")
job runs 3 hours a day    cloud =     360   own =    2000   cheaper: cloud
job runs 24 hours a day   cloud =    2880   own =    2000   cheaper: own

At 3 hours a day the cloud costs 360 against 2,000 for owning, because you stop paying when the job stops. Running all day, the cloud costs 2,880 and owning is cheaper. The break-even is about 500 hours a month in this example.

Cloud still has other advantages for a steady load (no hardware to maintain, easy to grow), but 'cloud is always cheaper' is false.

Cloud termPlain meaning
ElasticCapacity can grow and shrink with demand
Managed serviceThe provider runs the machines, patches and backups
Shared responsibilityThe provider secures the data center and platform. You secure your data, accounts and settings

Try a new scenario

A pipeline needs 10 powerful machines for 1 hour each night, and a big backfill once a quarter needs 100 machines for 6 hours. Why is renting a good fit here?

Work it out, then open the answer

Owning would mean buying 100 machines that sit idle almost all the time just to survive the quarterly peak. Renting lets you use 10 for an hour most days and 100 for 6 hours once a quarter, paying only for what runs. The unevenness of the load is what makes renting attractive.

Interview: what does the shared responsibility model mean?

The provider secures the underlying infrastructure, such as buildings, hardware and the platform. You secure what you put on it: your data, accounts, permissions and settings. Most breaches come from the customer's side of that line.

Close the lesson and explain it

Without scrolling up, explain when renting capacity beats owning it, and when it does not.

Practicals · load into the editor

After you read the theory, run these in the pane on the right. They execute in this tab, no cluster.

Finished this lesson?

Mark it as read when you could explain it to someone else.

Rate:
Was this useful?