Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing

dbt & Analytics Engineering

Progress0/13
x

Analytics Engineering

  • What is analytics engineering?12m
  • dbt project structure12m
  • Jinja basics14m

What dbt Is

  • What dbt is12m
  • source() and ref() as compiled names12m

Staging

  • Staging models14m
  • not_null tests as SQL12m

unique and relationships

  • unique tests12m
  • relationships tests14m

Marts and Incremental

  • Marts from staging CTEs14m
  • Incremental vs full refresh12m
  • Views, tables, incrementals10m

Capstone

  • Capstone: staging, mart, test16m
Back to track
  1. Learn
  2. dbt & Analytics Engineering
  3. What dbt Is
  4. What dbt is

Lesson 4 of 13 · Theory first, then run it

What dbt is

dbtsqlbeginner12 min

Overview

dbt is a compiler and a test runner around SELECT statements. The SQL editor runs compiled SELECTs. Nothing in this tab will run `dbt run`.

On this page7 sections›
  1. 1The idea
  2. 2Why this exists
  3. 3Picture this
  4. 4A small example
  5. 5Common beginner questions
  6. 6What comes next
  7. 7Practice

The idea

dbt (data build tool) is software that lets you transform data in your warehouse using SQL. You write SELECT statements, and dbt handles everything else: running them in the right order, replacing table names automatically, testing the results, and documenting what each model does.

Before dbt, data teams wrote SQL scripts and ran them manually or with custom bash scripts. Nobody knew which query depended on which, tests were an afterthought, and a rename rippled through dozens of files. dbt solves all of these problems by treating SQL transformations as a proper software project with dependencies, tests, and version control.

This tab runs the warehouse half only: the SQL editor executes compiled SELECT statements. There is no dbt CLI, profiles.yml, or docs site here. You write the SQL that dbt would generate, practicing the mental model before touching the tool.

Why this exists

Imagine a data team with 200 SQL queries that build reports. A table gets renamed from 'orders' to 'raw_orders'. Without dbt, someone must find and update every query that references that table. With dbt, you update one source definition, and every downstream model automatically uses the new name because they reference the model, not the raw table.

dbt also runs tests after every build. If a primary key has duplicates, dbt catches it before the dashboard shows wrong numbers. Without automated tests, data bugs reach stakeholders and erode trust in the data team.

Picture this

Analytics engineering is the discipline of turning messy source data into clean, tested, documented datasets that analysts can trust. dbt is the primary tool for this job. It works in three stages:

Compile, execute, test
Model (SELECT + Jinja)Compile (resolve refs)Execute SQL in warehouseTest results

dbt compiles Jinja templates into plain SQL, sends the SQL to your warehouse, then runs tests on the results.

  1. You write a model: a SELECT statement in a .sql file, optionally using Jinja templates like {{ ref('stg_orders') }}.
  2. dbt compiles the model: replaces ref() with actual table names and resolves Jinja logic.
  3. dbt executes the compiled SQL against your warehouse (BigQuery, Snowflake, Postgres, etc.).
  4. dbt runs tests: queries that check for nulls, duplicates, and orphan keys. Zero rows returned means the test passed.

A dbt project has a clear structure: models/ contains your SQL files, tests/ or schema.yml defines your tests, and dbt_project.yml configures the project.

A small example

Run the example below in this tab. Read the input, follow the code, then check the output matches what you expect.

SQLJinja template vs compiled SQL
-- What you write (dbt model with Jinja):
-- models/staging/stg_orders.sql
SELECT
    order_id,
    user_id AS customer_id,
    CAST(created_at AS DATE) AS order_date,
    order_total AS amount
FROM {{ source('jaffle', 'orders') }}

-- What dbt compiles it to (plain SQL):
SELECT
    order_id,
    user_id AS customer_id,
    CAST(created_at AS DATE) AS order_date,
    order_total AS amount
FROM jaffle.orders
dbt conceptIn production (dbt CLI)In this tab
ModelSELECT in a .sql file with ref()/source()SELECT in the SQL editor (already compiled)
Compiledbt CLI resolves Jinja to plain SQLYou write the compiled SQL directly
Testdbt test runs zero-row queriesYou run those same queries manually
ref()dbt resolves to the actual table nameYou write the table name yourself
source()Points to a raw table you do not ownYou write FROM table_name directly
SQLStaging-shaped SELECT for this tab
-- A simple staging model (compiled SQL, no Jinja)
SELECT
    order_id,
    user_id AS customer_id,
    order_total AS amount
FROM orders
LIMIT 8;

Common beginner questions

Why do I need dbt if I already know SQL?

dbt does not replace SQL. It organizes your SQL into a maintainable project. Without dbt, you have a folder of scripts with no dependency tracking, no testing, and no documentation. dbt adds all three.

Is dbt an ETL tool?

dbt handles only the T (Transform). It does not extract data from sources or load it into the warehouse. Other tools (Fivetran, Airbyte, custom scripts) handle E and L. dbt transforms data that is already in the warehouse.

Do I write Python or SQL?

dbt models are primarily SQL. dbt also supports Python models (using pandas or PySpark), but most teams use SQL for the majority of transformations because warehouses are optimized for SQL execution.

You already built a star schema in SQL

The SQL track taught you to model facts and dimensions. dbt versions that same SQL, tests it automatically, and chains models with ref() instead of hardcoded table names.

Leave Jinja out of this tab

Writing {{ ref('stg_orders') }} in the SQL editor will cause a syntax error. This tab runs plain SQL only. You practice the compiled output that dbt would generate.

What comes next

The next lesson introduces source() and ref(): the two functions that build the dependency graph between models. Understanding these is the key to understanding how dbt knows the correct execution order.

Practice

Run Sample to preview a staging-shaped SELECT. Then complete the exercise: select order_id, customer_id, and amount from orders.

Practicals · load into the editor

After you read the theory, run these in the pane on the right. They execute in this tab, no cluster.

Rate:
Was this useful?
Jinja basicssource() and ref() as compiled names