Learn about the reasons behind dbt's existence and how it's used to create data transformation projects.

Foundations of dbt

This is the code editor window. The terminal window is the panel below.

Run the following command in the terminal area:
  
dbt

Running the dbt command

Introducing dbt

Version of dbt

Getting help

Options

Likely to use dbt

Unlikely to use dbt

Users of dbt

Creating a dbt project

Project Name: nyc_yellow_taxi
Database type: duckdb


Run the following commands in the terminal window:


dbt init


Note: You will need to specify the project name (nyc_yellow_taxi) and
select the database type ([1] duckdb)
  
Once complete run:
  
cd nyc_yellow_taxi

ls


Initializing a dbt project

# Modify the project name
____:
  outputs:
    dev:
      # Change the database type
      type: ____
      path: dbt.duckdb
  target: dev


nyc_yellow_taxi:
  outputs:
    dev:
      type: duckdb
      path: dbt.duckdb
  target: dev
  



# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'taxi_project'
version: '1.0.0'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
profile: 'nyc_yellow_taxi'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  # directory which will store compiled SQL files
clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"


# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
  nyc_yellow_taxi:
  # Config indicated by + and applies to all files under models/example/
    taxi_rides:
    +materialized: view


{{ config(materialized='view')}}


with src_data as (
    select * from read_parquet('yellow_tripdata_2023-01-partial.parquet')
)

select * from source_data


nyc_yellow_taxi:
  outputs:
    dev:
      type: duckdb
      path: dbt.duckdb
  target: dev


Run the following command in the terminal after editing the profiles.yml file:
  
dbt debug


Creating a project profile

Welcome to dbt

Get into the true substance of dbt by working with data models and creating SQL based models within dbt. Learn about basic SQL models, define dependencies, update, and troubleshoot various model issues. 

Working with a first project

dbt project workflow

Running a project

What is a dbt model?

Features of a data model

Creating a dbt model

Updating dbt models

Config files

Updating a dbt model

dbt projects and models

Learn how to use Jinja templates to simplify dbt models and define hierarchical dependencies with the `ref` function to ensure correct model execution.

Creating and generating dbt documentation

dbt docs Command Options

dbt documentation flow

Creating dbt documentation

Jinja templates

What can Jinja do?

Process a Jinja template

Hierarchical models in dbt

Hierarchical model creation

Updating model hierarchies

More on dbt models

This course introduces you to dbt (data build tool), a popular tool that has given rise to analytics engineering for the creation and maintenance of extract, transform, and load (ELT) pipelines. You'll learn to build data warehouses, perform data modeling and transformations, and design tests to perform data validation. You'll use SQL and shell commands to perform a variety of tasks with dbt, along with discovering how to generate documentation for your warehouse users!

dbt, or the data build tool, has taken the data world by storm. This course introduces you to the concepts, terminology, and methods of using dbt to implement an example data warehouse. You'll gain an understanding of what dbt is, when it should be used, and best practices when implementing data warehousing. You will work with real datasets and perform extract, load, and transform operations as implemented in data analyst, data engineering, and analytics engineering roles.

Learners will develop the skills to define a data warehouse from scratch, model and transform data, and build tests! This course will provide you with a solid foundation to build upon in your dbt journey, regardless of the type of data warehouse you intend to implement. Gain confidence about how and when to use dbt by working through exercises using SQL and shell commands.

Data Warehousing Concepts

Introduction to Shell

Joining Data in SQL

This course introduces dbt for data modeling, transformations, testing, and building documentation.

Running the dbt command

Introduction to dbt

Praktische interactieve oefening