从零开始练习搭建一个 dbt 项目。将不同类型的数据加载到 dbt 项目中，并设置多种分层的 dbt 暂存模型。本章聚焦于 ELT 流程中的 E 和 L 部分。

dbt 案例学习简介

检查 dbt 安装

初始化 dbt 项目

设置 dbt 项目并加载数据

将配送中心作为 dbt 种子文件加载

创建配送中心的分层（staging）模型

通过 dbt source 加载并创建用户模型

进一步练习 dbt source 与 staging 模型

调试常见错误

设置 dbt

深入掌握 dbt 的数据建模。从初步的暂存模型逐步构建到最终的数据集市模型，以回答关键业务需求。过程中，您还将练习创建数据测试，以防止数据质量漂移并提供保护。

创建 dbt 描述与测试

添加 dbt 列描述

添加 dbt 列级测试

构建 dbt 数据集市与快照模型

复习：dbt 模型层级

为 customers 创建数据集市模型

为商品创建数据集市模型

调试更多错误

构建 dbt 快照模型

构建 dbt 模型

学习如何用 Jinja 增强您的 dbt 项目，从而精简代码、减少冗余并提升可维护性。您将练习使用变量、循环和宏，构建更高效的转换工作流。

在 dbt 中介绍 Jinja

looker_ecommerce:
  outputs:
    dev:
      type: duckdb
      path: /home/repl/dbt.duckdb
  target: dev



# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'looker_ecommerce'
version: '1.0.0'

# This setting configures which "profile" dbt uses for this project.
profile: 'looker_ecommerce'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"


version: 2

seeds:
  - name: looker__distribution_centers


version: 2

sources:
  - name: looker_ecommerce
    tables:
      - name: users
      - name: events
      - name: inventory_items
      - name: order_items
      - name: orders
      - name: products

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
        data_tests:
          - unique
          - not_null          
      - name: name 
        description: Distribution center name
        data_tests:
          - not_null
      - name: latitude
        description: Distibution center location latitude
      - name: longitude
        description: Distribution center location longitude

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
      - name: first_name
        description: User first name
        data_tests:
          - not_null
      - name: last_name
        description: User last name
        data_tests:
          - not_null
      - name: email
        description: User email
        data_tests:
          - not_null
      - name: age
        description: User age in years
        data_tests:
          - not_null
      - name: gender
        description: User gender
        data_tests:
          - accepted_values:
              values: ['F', 'M', null]
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: created_at
        description: User created at timestamp
        data_tests:
          - not_null

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
        data_tests:
          - not_null
      - name: created_at
        description: Event created at timestamp
        data_tests:
          - not_null
      - name: ip_address        
        description: Event IP address
        data_tests:
          - not_null
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
        data_tests:
          - accepted_values:
              values: ['Chrome', 'Firefox', 'Safari', 'IE', 'Other'] 
      - name: traffic_source
        description: Traffic source for event information
        data_tests:
          - not_null
          - accepted_values:
              values: ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube']        
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
        data_tests:
          - unique
          - not_null
      - name: product_id
        description: Product ID associated with the inventory
        data_tests:
          - not_null
      - name: created_at
        description: Inventory created at timestamp 
        data_tests:
          - not_null
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored 

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
        data_tests:
          - unique
          - not_null
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID associated with the order
        data_tests:
          - not_null        
      - name: status
        description: Order status 
        data_tests:
          - not_null        
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products
    description: Product information such as SKU and category
    columns: 
      - name: id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
      - name: distribution_center_id
        description: Distribution center ID where the product is stored 
        data_tests:
          - relationships:
              to: ref('stg_looker__distribution_centers')
              field: id            

SELECT 
  id,
  name,
  latitude,
  longitude
FROM {{ ref('looker__distribution_centers') }}

SELECT
  id,
  user_id,
  sequence_number,
  session_id,
  created_at,
  ip_address,
  city,
  state,
  postal_code,
  browser,
  traffic_source,
  uri,
  event_type
FROM {{ source('looker_ecommerce', 'events') }}

SELECT
  id,
  product_id,
  created_at,
  sold_at,
  cost,
  product_category,
  product_name,
  product_brand,
  product_retail_price,
  product_department,
  product_sku,
  product_distribution_center_id
FROM {{ source('looker_ecommerce', 'inventory_items') }}

SELECT
  id,
  order_id,
  user_id,
  product_id,
  inventory_item_id,
  status,
  created_at,
  shipped_at,
  delivered_at,
  returned_at,
  sale_price
FROM {{ source('looker_ecommerce', 'order_items') }}

SELECT 
  order_id,
  user_id,
  status,
  gender,
  created_at,
  returned_at,
  shipped_at,
  delivered_at,
  num_of_item
FROM {{ source('looker_ecommerce', 'orders') }}

SELECT
  id,
  cost,
  category,
  name,
  brand,
  retail_price,
  department,
  sku,
  distribution_center_id
FROM {{ source('looker_ecommerce', 'products') }}

SELECT 
  id,
  first_name,
  last_name,
  email,
  age,
  gender,
  state,
  street_address,
  postal_code,
  city,
  country,
  latitude,
  longitude,
  traffic_source,
  created_at
FROM {{ source('looker_ecommerce', 'users') }}

{% snapshot inventory_snapshot %}

{{
    config(
      target_schema='main',
      unique_key='id',
      strategy='timestamp',
      updated_at='created_at' 
    )
}}

SELECT * 
FROM {{ source('looker_ecommerce', 'inventory_items') }}

{% endsnapshot %}


{% snapshot orders_snapshot %}

{{
    config(
      target_schema='main',
      unique_key='order_id',
      strategy='timestamp',
      updated_at='created_at' 
    )
}}

SELECT * 
FROM {{ source('looker_ecommerce', 'orders') }}

{% endsnapshot %}


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
        data_tests:
          - not_null
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,
    pb.product_name,
    pb.product_category,
    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


用 dbt compile 热身

version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
        data_tests: # Remove not_null data test
          - not_null # Remove not_null data test
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


用 dbt compile 热身 2

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM stg_looker__users -- Fix Jinja reference
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM stg_looker__orders -- Fix Jinja reference
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }} 
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,
    pb.product_name,
    pb.product_category,
    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }} -- Fix Jinja reference
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }} -- Fix Jinja reference
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


使用 dbt compile 进行调试

-- Set the value to 'Complete'
{% set order_items_status = ____ %}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
	{# Pass in the variable order_items_status #}
    WHERE status = '{{ ____ }}'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


-- Set the value to 'Complete'
{% set order_items_status = 'Complete' %}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
  	{# Pass in the variable order_items_status #}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


使用 dbt Jinja 设置变量

{% set order_items_status = 'Complete' %}
{# Set a Jinja variable called country with value 'Australia' #}
____ 

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
	{# Replace with jinja dynamic filtering #}
    WHERE country = 'Australia' 
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


{% set order_items_status = 'Complete' %}
{# Set a Jinja variable called country with value 'Australia' #}
{% set country = 'Australia' %}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
 	{# Replace with jinja dynamic filtering #}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


使用 dbt Jinja 2 设置变量

在 dbt 中更复杂的 Jinja 用法

{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
	-- Replace the following logic with a SELECT * FROM {{ ref('int_orders__pivoted') }}
	SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT CASE WHEN status = 'Shipped' THEN order_id END) AS num_orders_Shipped,
        COUNT(DISTINCT CASE WHEN status = 'Complete' THEN order_id END) AS num_orders_Complete,
        COUNT(DISTINCT CASE WHEN status = 'Processing' THEN order_id END) AS num_orders_Processing,
        COUNT(DISTINCT CASE WHEN status = 'Cancelled' THEN order_id END) AS num_orders_Cancelled,
        COUNT(DISTINCT CASE WHEN status = 'Returned' THEN order_id END) AS num_orders_Returned
  	FROM {{ ref('stg_looker__orders') }}
    -- Remember to remove the GROUP BY
  	GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,

        COUNT(DISTINCT CASE WHEN traffic_source = 'Adwords' THEN user_id END) AS num_traffic_source_Adwords,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Email' THEN user_id END) AS num_traffic_source_Email,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Facebook' THEN user_id END) AS num_traffic_source_Facebook,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Organic' THEN user_id END) AS num_traffic_source_Organic,
        COUNT(DISTINCT CASE WHEN traffic_source = 'YouTube' THEN user_id END) AS num_traffic_source_YouTube,

        COUNT(DISTINCT CASE WHEN browser = 'Chrome' THEN user_id END) AS num_browser_Chrome,
        COUNT(DISTINCT CASE WHEN browser = 'Firefox' THEN user_id END) AS num_browser_Firefox,
        COUNT(DISTINCT CASE WHEN browser = 'Safari' THEN user_id END) AS num_browser_Safari,
        COUNT(DISTINCT CASE WHEN browser = 'IE' THEN user_id END) AS num_browser_IE,
        COUNT(DISTINCT CASE WHEN browser = 'Other' THEN user_id END) AS num_browser_Other      
        FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


-- Fill in order status values in line 2
{% set order_statuses = ____ %} 

SELECT 
    user_id,
    COUNT(DISTINCT order_id) AS num_orders,
    -- Fill in lines 7 - 13 for looping through creating orders statuses pivot columns
    {%- ____ order_status ____ order_statuses %}
    COUNT(DISTINCT CASE WHEN status = '{{ order_status }}' THEN order_id END) AS num_orders_{{ order_status }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- ____ %}
FROM {{ ref('stg_looker__orders') }}
GROUP BY 1

-- Fill in order status values in line 2
{% set order_statuses = ['Shipped', 'Complete', 'Processing', 'Cancelled', 'Returned'] %} 

SELECT 
    user_id,
    COUNT(DISTINCT order_id) AS num_orders,
    -- Fill in lines 7 - 13 for looping through creating orders statuses pivot columns
    {%- for order_status in order_statuses %}
    COUNT(DISTINCT CASE WHEN status = '{{ order_status }}' THEN order_id END) AS num_orders_{{ order_status }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}
FROM {{ ref('stg_looker__orders') }}
GROUP BY 1

{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
	-- Replace the following logic with a SELECT * FROM {{ ref('int_orders__pivoted') }}
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
  	-- Remember to remove the GROUP BY
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,

        COUNT(DISTINCT CASE WHEN traffic_source = 'Adwords' THEN user_id END) AS num_traffic_source_Adwords,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Email' THEN user_id END) AS num_traffic_source_Email,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Facebook' THEN user_id END) AS num_traffic_source_Facebook,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Organic' THEN user_id END) AS num_traffic_source_Organic,
        COUNT(DISTINCT CASE WHEN traffic_source = 'YouTube' THEN user_id END) AS num_traffic_source_YouTube,

        COUNT(DISTINCT CASE WHEN browser = 'Chrome' THEN user_id END) AS num_browser_Chrome,
        COUNT(DISTINCT CASE WHEN browser = 'Firefox' THEN user_id END) AS num_browser_Firefox,
        COUNT(DISTINCT CASE WHEN browser = 'Safari' THEN user_id END) AS num_browser_Safari,
        COUNT(DISTINCT CASE WHEN browser = 'IE' THEN user_id END) AS num_browser_IE,
        COUNT(DISTINCT CASE WHEN browser = 'Other' THEN user_id END) AS num_browser_Other      
        FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


在重复 SQL 上使用 Jinja for 循环 1

{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
)

, web_traffic AS (
  	-- Replace the following logic with a SELECT * FROM {{ ref('int_events__pivoted') }}
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,

        COUNT(DISTINCT CASE WHEN traffic_source = 'Adwords' THEN user_id END) AS num_traffic_source_Adwords,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Email' THEN user_id END) AS num_traffic_source_Email,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Facebook' THEN user_id END) AS num_traffic_source_Facebook,
        COUNT(DISTINCT CASE WHEN traffic_source = 'Organic' THEN user_id END) AS num_traffic_source_Organic,
        COUNT(DISTINCT CASE WHEN traffic_source = 'YouTube' THEN user_id END) AS num_traffic_source_YouTube,

        COUNT(DISTINCT CASE WHEN browser = 'Chrome' THEN user_id END) AS num_browser_Chrome,
        COUNT(DISTINCT CASE WHEN browser = 'Firefox' THEN user_id END) AS num_browser_Firefox,
        COUNT(DISTINCT CASE WHEN browser = 'Safari' THEN user_id END) AS num_browser_Safari,
        COUNT(DISTINCT CASE WHEN browser = 'IE' THEN user_id END) AS num_browser_IE,
        COUNT(DISTINCT CASE WHEN browser = 'Other' THEN user_id END) AS num_browser_Other      
        FROM {{ ref('stg_looker__events') }}
    -- Remember to remove the GROUP BY
    GROUP BY 1
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


{% set traffic_source_values = ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube'] %}
{% set browser_values = ['Chrome', 'Firefox', 'Safari', 'IE', 'Other'] %}

SELECT 
    user_id,
    COUNT(DISTINCT session_id) AS num_web_sessions,
        
    -- Fill in lines 9 - 11 for looping through traffic source pivot
    {%- for ____ in traffic_source_values %}
    COUNT(DISTINCT CASE WHEN traffic_source = '{{ ____ }}' THEN user_id END) AS num_traffic_source_{{ ____ }},
    {%- endfor %}
      
	-- Fill in lines 14 - 19 for looping through browser pivot
    {%- for ____ in browser_values %}
    COUNT(DISTINCT CASE WHEN browser = '{{ ____ }}' THEN user_id END) AS num_browser_{{ ____ }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- ____ %}

FROM {{ ref('stg_looker__events') }}
GROUP BY 1

{% set order_statuses = ['Shipped', 'Complete', 'Processing', 'Cancelled', 'Returned'] %}

SELECT 
    user_id,
    COUNT(DISTINCT order_id) AS num_orders,
    {%- for order_status in order_statuses %}
    COUNT(DISTINCT CASE WHEN status = '{{ order_status }}' THEN order_id END) AS num_orders_{{ order_status }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}
FROM {{ ref('stg_looker__orders') }}
GROUP BY 1

{% set traffic_source_values = ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube'] %}
{% set browser_values = ['Chrome', 'Firefox', 'Safari', 'IE', 'Other'] %}

SELECT 
    user_id,
    COUNT(DISTINCT session_id) AS num_web_sessions,
    
    -- Fill in lines 9 - 11 for looping through traffic source pivot    
    {%- for traffic_source_value in traffic_source_values %}
    COUNT(DISTINCT CASE WHEN traffic_source = '{{ traffic_source_value }}' THEN user_id END) AS num_traffic_source_{{ traffic_source_value }},
    {%- endfor %}
    
	-- Fill in lines 14 - 19 for looping through browser pivot
    {%- for browser_value in browser_values %}
    COUNT(DISTINCT CASE WHEN browser = '{{ browser_value }}' THEN user_id END) AS num_browser_{{ browser_value }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}      

FROM {{ ref('stg_looker__events') }}
GROUP BY 1

{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
)

, web_traffic AS (
  	-- Replace the following logic with a SELECT * FROM {{ ref('int_events__pivoted') }}
    SELECT *
  	FROM {{ ref('int_events__pivoted') }}
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


在重复 SQL 上使用 Jinja 循环 2

version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
        data_tests:
          - not_null        
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
        data_tests:
          - not_null        
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold
        data_tests:
          - not_null        


{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
)

, web_traffic AS (
    SELECT *
    FROM {{ ref('int_events__pivoted') }}
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    WHERE sold_at IS NULL
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,
    pb.product_name,
    pb.product_category,
    
    {{ coalesce_and_round('oi.sales_amount', 2) }} AS sales_amount,
    
    -- Fill in the coalesce and rounding of the cost of goods sold to 2 decimals
	____ AS cost_of_goods_sold, 
    
    {{ coalesce_and_round('oi.sales_amount', 0) }} - {{ coalesce_and_round('ii.cost_of_goods_sold', 0) }} AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


{% set traffic_source_values = ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube'] %}
{% set browser_values = ['Chrome', 'Firefox', 'Safari', 'IE', 'Other'] %}

SELECT 
    user_id,
    COUNT(DISTINCT session_id) AS num_web_sessions,
    
    {#- jinja loop for pivoting out traffic source values -#}
    {%- for traffic_source_value in traffic_source_values %}
    COUNT(DISTINCT CASE WHEN traffic_source = '{{ traffic_source_value }}' THEN user_id END) AS num_traffic_source_{{ traffic_source_value }},
    {%- endfor %}

    {#- jinja loop for pivoting out browser values -#}
    {%- for browser_value in browser_values %}
    COUNT(DISTINCT CASE WHEN browser = '{{ browser_value }}' THEN user_id END) AS num_browser_{{ browser_value }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}

FROM {{ ref('stg_looker__events') }}
GROUP BY 1

WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    WHERE sold_at IS NULL
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,
    pb.product_name,
    pb.product_category,

    {{ coalesce_and_round('oi.sales_amount', 2) }} AS sales_amount,

    -- Fill in the coalesce and rounding of the cost of goods sold to 2 decimals
    {{ coalesce_and_round('ii.cost_of_goods_sold', 2) }} AS cost_of_goods_sold,
    
    {{ coalesce_and_round('oi.sales_amount', 0) }} - {{ coalesce_and_round('ii.cost_of_goods_sold', 0) }} AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


-- Copy paste the Jinja macro here
{%- macro coalesce_and_round(column_name, decimal_places = 2) -%}
    ROUND(COALESCE({{ column_name }}, 0), {{ decimal_places }})
{%- endmacro %}

结合 Jinja 的宏

{# Modify lines 2 and 3 to trim whitespaces #}
{% set order_items_status = 'Complete' %}
{% set country = 'Australia' %}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
)

, web_traffic AS (
    SELECT *
  	FROM {{ ref('int_events__pivoted') }}
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


SELECT 
    user_id,
    COUNT(DISTINCT session_id) AS num_web_sessions,
    
    {#- jinja loop for pivoting out traffic source values -#}
    {%- for traffic_source_value in traffic_source_values %}
    COUNT(DISTINCT CASE WHEN traffic_source = '{{ traffic_source_value }}' THEN user_id END) AS num_traffic_source_{{ traffic_source_value }},
    {%- endfor %}

    {#- jinja loop for pivoting out browser values -#}
    {%- for browser_value in browser_values %}
    COUNT(DISTINCT CASE WHEN browser = '{{ browser_value }}' THEN user_id END) AS num_browser_{{ browser_value }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}

FROM {{ ref('stg_looker__events') }}
GROUP BY 1
      

{% set order_statuses = ['Shipped', 'Complete', 'Processing', 'Cancelled', 'Returned'] %}

SELECT 
    user_id,
    COUNT(DISTINCT order_id) AS num_orders,
    {%- for order_status in order_statuses %}
    COUNT(DISTINCT CASE WHEN status = '{{ order_status }}' THEN order_id END) AS num_orders_{{ order_status }}
    {%- if not loop.last -%}
        ,
    {%- endif -%}
    {%- endfor %}
FROM {{ ref('stg_looker__orders') }}
GROUP BY 1


{# Modify lines 2 and 3 to trim whitespaces #}
{%- set order_items_status = 'Complete' -%}
{%- set country = 'Australia' -%}

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
    WHERE country = '{{ country }}'
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = '{{ order_items_status }}'
    GROUP BY 1
)

, orders AS (
    SELECT * 
    FROM {{ ref('int_orders__pivoted') }}
)

, web_traffic AS (
    SELECT *
  	FROM {{ ref('int_events__pivoted') }}
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.* EXCLUDE(user_id),
    wt.* EXCLUDE(user_id)
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


使用 Jinja 去除空白字符

回顾：dbt 案例研究

用 Jinja 改进 dbt

通过我们的动手实践型 dbt（Data Build Tool）案例研究课程，释放数据转换的力量。本课程面向 dbt 初学到入门阶段的用户，聚焦实用的真实场景练习，为您提供多次机会去构建、转换并加深对核心 dbt 命令的理解。通过亲手实践，您将在流程的每一步建立信心——从搭建 dbt 环境起步，到创建能够解决关键业务问题的高级模型。通过交互式练习，您将积累处理真实 ETL 管道项目所需的经验与信心。

使用 dbt（数据构建工具）将原始数据转化为干净、可靠的模型——这是一款现代的、基于 SQL 的转换框架，全球各地的数据团队都在使用。 这门实战案例课程专为初学者设计，帮助你通过循序渐进的实践练习，掌握真实场景中的实用技能。 你将搭建自己的 dbt 环境，大规模建模数据，并使用 dbt 的内置功能编写可复用代码。<br><br><h2>设置你的项目并探索数据</h2>首先搭建一个 dbt 项目，并使用真实的电子商务数据集进行实践。 你将对原始数据进行结构化处理，配置配置文件，并调试语法问题，同时深入了解每个转换步骤背后的业务背景。<br><br><h2>构建并验证模型</h2>学习创建可扩展的暂存模型，并应用数据质量检查，确保您的数据集准确且可直接用于分析。 你将打下坚实基础，学会回答关键的业务问题。<br><br><h2>使用 Jinja 实现自动化</h2>通过学习如何使用 Jinja 编写可复用、易维护的代码，完成本课程。 你将使用变量、控制流和循环来遵循 DRY（不要重复自己）原则，从而简化你的 dbt 工作流程。

Data Manipulation in SQL

Intermediate dbt

通过实践练习，学习使用dbt将原始数据转化为清晰可靠的模型。

案例研究：使用 dbt 构建电商数据模型

通过动手实战，学习如何用 dbt 将原始数据转化为干净、可靠的模型。 

在 dbt 中更复杂的 Jinja 用法

Create Your Free Account