On this page
Imagine an online food delivery company. Every order, payment and delivery creates data. Someone has to move that data to a safe place and keep it correct. Someone has to find out which restaurants are growing. Someone has to build a model that predicts delivery times. Those are three different jobs, and they have three different names: data engineer, data analyst and data scientist. People mix the titles up all the time, partly because companies use them differently. The ideas below hold in most places.
Pipelines, tables, checks.
Success: the data is correct and on time.
Reports and dashboards.
Success: a better decision.
Statistics and models.
Success: a prediction you can trust.
The factory comparison
Think of a water supply. The data engineer builds the pipes, the tanks and the filters, and makes sure clean water arrives every day. The data analyst tests the water and reports on how much each area uses and where the trends are. The data scientist builds a system to predict next month's demand. If the pipes are bad, neither of the others can do their job, which is why teams with lots of data hire engineers early.
Data engineer
A data engineer builds and runs the systems that make data available and trustworthy.
- A day looks like: writing a pipeline that loads yesterday's orders, fixing a job that failed overnight, adding a check that finds duplicate rows, designing a table so reports run fast.
- Main skills: SQL, Python, data modeling, pipelines and scheduling, cloud storage and warehouses, and sometimes Spark or Kafka.
- Typical tools: a warehouse such as Snowflake or BigQuery, Airflow, dbt, Spark, Docker.
- Output: tables and pipelines that other people use. Success is that nobody has to think about them, because they work.
Learn it step by step with the data engineer roadmap for freshers.
Data analyst
A data analyst uses data to answer business questions.
- A day looks like: working out why sales dropped last week, building a dashboard for the marketing team, cleaning a spreadsheet, presenting three charts to a manager.
- Main skills: SQL, Excel or Google Sheets, a dashboard tool, some statistics, and above all clear explanation, because the work ends with a person making a decision.
- Typical tools: SQL, Excel, Power BI, Tableau, Looker, sometimes Python or R for deeper analysis.
- Output: reports, dashboards and recommendations. Success is a better decision.
Data scientist
A data scientist uses statistics and machine learning to predict, classify or explain.
- A day looks like: building a model that predicts which customers will leave, testing whether a new feature really increased orders, preparing data for a model, checking how accurate it is.
- Main skills: Python or R, statistics and probability, machine learning, experiment design, plus enough SQL to fetch the data.
- Typical tools: Python with pandas and scikit-learn, notebooks, and machine learning libraries.
- Output: models and experiment results. Success is a prediction that is useful and accurate enough to trust.
Side by side
| Data engineer | Data analyst | Data scientist | |
|---|---|---|---|
| Main question | How do we get reliable data to people? | What happened, and why? | What will happen, and what should we do? |
| Core skills | SQL, Python, modeling, pipelines, cloud | SQL, spreadsheets, dashboards, statistics | Python or R, statistics, machine learning |
| Works closest with | Software engineers and analysts | Business teams and managers | Product teams and analysts |
| Builds | Pipelines and data tables | Reports and dashboards | Models and experiments |
| Success looks like | Data that is correct and on time | A good decision | A trustworthy prediction |
| Maths needed | Light | Moderate | Heavy |
Where they overlap
The boundaries are blurry. In a small company one person may do all 3 jobs. In a large company each has a team. A few things help you read a job post correctly:
- An analyst who writes a lot of SQL transformations is doing analytics engineering, which is close to data engineering.
- A data scientist who spends most of the time cleaning data and building pipelines for models is working like a data engineer.
- A data engineer who builds model-serving infrastructure is working next to machine learning, often called an ML engineer.
The title tells you less than the list of skills in the post, so read the requirements, not the name.
Which one should a fresher choose?
Ask what you enjoy doing:
- You like building things that run, fixing why they broke, and writing code: data engineering.
- You like finding the story in numbers, and explaining it to people: data analysis.
- You like maths, statistics and experimenting with models: data science.
Do not worry about getting this perfect. All three begin with the same skill, SQL, so starting with SQL for beginners keeps every option open. After that, Python is useful for all three. Many people start as analysts and move toward engineering or science as they find what they like, and the move from analyst to engineer is a well-worn one, since the extra skills are modeling, pipelines and cloud.
If you decide on data engineering, the Learn tracks start with SQL and Python, and the roadmap shows how the rest fits together. For everyday vocabulary, read what is a data pipeline, because it is the thing data engineers build.