Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing

dbt & Analytics Engineering

Progress0/13
x

Analytics Engineering

  • What is analytics engineering?12m
  • dbt project structure12m
  • Jinja basics14m

What dbt Is

  • What dbt is12m
  • source() and ref() as compiled names12m

Staging

  • Staging models14m
  • not_null tests as SQL12m

unique and relationships

  • unique tests12m
  • relationships tests14m

Marts and Incremental

  • Marts from staging CTEs14m
  • Incremental vs full refresh12m
  • Views, tables, incrementals10m

Capstone

  • Capstone: staging, mart, test16m
Back to track
  1. Learn
  2. dbt & Analytics Engineering
  3. What dbt Is
  4. source() and ref() as compiled names

Lesson 5 of 13 · Theory first, then run it

source() and ref() as compiled names

dbtsqlbeginner12 min

Overview

{{ source('jaffle', 'orders') }} becomes orders. {{ ref('stg_orders') }} becomes stg_orders. You write the compiled form.

On this page7 sections›
  1. 1What you will do
  2. 2Why this skill
  3. 3How the code works
  4. 4Worked examples
  5. 5Common beginner questions
  6. 6What comes next
  7. 7Practice

What you will do

In dbt, source() and ref() are the two functions that connect models together. source() points to a raw table that exists in your warehouse but is not managed by dbt (like a table loaded by Fivetran). ref() points to another dbt model (a SQL file in your project).

When dbt compiles your project, it replaces source('schema', 'table') with the actual fully-qualified table name, and ref('model_name') with the table or view that model creates. This replacement is how dbt builds its dependency graph: it knows model B depends on model A because B contains ref('A').

In this tab, you perform that substitution yourself. You write the compiled SQL directly (FROM orders instead of FROM {{ source('jaffle', 'orders') }}).

Why this skill

Imagine a project with 50 dbt models. Model stg_orders reads from the raw orders table. Models fct_revenue, fct_refunds, and dim_customers all read from stg_orders. If you hardcode the table name 'stg_orders' in all three downstream models and later rename stg_orders to stg_sales_orders, you must update three files. With ref('stg_orders'), you rename once and dbt handles the rest.

More importantly, ref() tells dbt the execution order. dbt knows to build stg_orders before fct_revenue because fct_revenue refs stg_orders. Without ref(), dbt would not know the dependency exists and might try to build fct_revenue before its input is ready.

How the code works

source() points to raw, ref() points to models
source('jaffle','orders')ref('stg_orders')ref('fct_revenue')

Sources are raw tables you do not own. Refs are models you build. The arrows create the DAG.

Here is how a three-layer project uses source() and ref():

Worked examples

Run the example below in this tab. Read the input, follow the code, then check the output matches what you expect.

SQLsource() becomes a fully-qualified table name
-- Layer 1: Staging model reads from a source
-- models/staging/stg_orders.sql (what you write)
SELECT order_id, order_total AS gmv, order_status
FROM {{ source('jaffle', 'orders') }}
WHERE order_status = 'paid'

-- Compiled to:
SELECT order_id, order_total AS gmv, order_status
FROM jaffle.orders
WHERE order_status = 'paid'
SQLref() becomes the actual table/view name
-- Layer 2: Mart model reads from a staging model
-- models/marts/fct_revenue.sql (what you write)
SELECT order_id, gmv
FROM {{ ref('stg_orders') }}

-- Compiled to:
SELECT order_id, gmv
FROM analytics.stg_orders

The dependency graph that dbt builds from these references:

ModelReads fromFunction useddbt execution order
stg_ordersRaw orders tablesource()Builds first
fct_revenuestg_orders modelref()Builds second
dim_customersstg_orders + stg_customersref() + ref()Builds after both staging models

A cycle of refs is invalid. If model A refs model B, and model B refs model A, dbt refuses to compile. This is the same 'no cycles' rule you learned in the orchestration track's DAG lesson.

SQLCompiled paid GMV query
-- Compiled SQL for this tab: paid orders GMV
SELECT order_id, order_total AS gmv
FROM orders
WHERE order_status = 'paid'
LIMIT 20;

Common beginner questions

Why not just write FROM orders directly?

In a production project, raw table names can change (schema renames, warehouse migrations). source() centralizes the mapping. For model-to-model references, ref() provides dependency tracking. In this practice tab, you write compiled SQL for simplicity, but understand that production code uses source() and ref().

Can a staging model ref another staging model?

Technically yes, but it is discouraged. Staging models should read directly from sources, doing only rename/cast/filter. Cross-referencing between staging models creates confusing dependency chains.

Never source() from a mart

Sources are raw tables from external systems. If you need data from another dbt model, use ref(). source() and ref() serve different purposes.

The DAG from orchestration applies here

dbt's ref() graph is exactly the DAG you implemented in the orchestration track. dbt uses topological sort to determine build order, just like the topo_order function you wrote.

What comes next

Now that you understand how models connect, the next lesson teaches staging models: the first transformation layer where you rename columns, cast types, and filter junk from raw sources.

Practice

Write the compiled paid GMV query: select order_id and order_total as gmv from orders where status is paid. Limit 20.

Practicals · load into the editor

After you read the theory, run these in the pane on the right. They execute in this tab, no cluster.

Rate:
Was this useful?
What dbt isStaging models