Pratique a construção de um projeto dbt do zero. Aplique suas habilidades para carregar diferentes tipos de dados no projeto dbt e configurar uma variedade de modelos de staging. Este capítulo foca nas partes E e L do processo ELT.

Apresentando o estudo de caso de dbt

Verificar a instalação do dbt

Inicializar o projeto dbt

Configurando o projeto dbt e carregando dados

Carregar o centro de distribuição como um arquivo seed do dbt

Criar modelo de stage para centros de distribuição

Carregar e criar modelo de usuários via dbt source

Mais prática com fontes e modelos de staging no dbt

Depure erros comuns

Configurando o dbt

Aprofunde-se na modelagem de dados com dbt. Construa o pipeline de dados desde os modelos de staging iniciais até os modelos finais de data mart para responder a necessidades críticas do negócio. Ao longo do caminho, ganhe experiência criando testes de dados para evitar a degradação da qualidade dos dados.

Criando descrições e testes no dbt

looker_ecommerce:
  outputs:
    dev:
      type: duckdb
      path: /home/repl/dbt.duckdb
  target: dev



# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'looker_ecommerce'
version: '1.0.0'

# This setting configures which "profile" dbt uses for this project.
profile: 'looker_ecommerce'

# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:         # directories to be removed by `dbt clean`
  - "target"
  - "dbt_packages"


version: 2

seeds:
  - name: looker__distribution_centers


version: 2

sources:
  - name: looker_ecommerce
    tables:
      - name: users
      - name: events
      - name: inventory_items
      - name: order_items
      - name: orders
      - name: products

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
      - name: name 
        description: Distribution center name
      - name: latitude
        description: Distribution center location latitude
      - name: ____ # Fill in the column name for longitude
        description: ____ # Fill in the column description

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
      - name: first_name
        description: User first name
      - name: last_name
        description: User last name
      - name: email
        description: User email
      - name: age
        description: User age in years
      - name: gender
        description: User gender
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
      - name: created_at
        description: User created at timestamp

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
      - name: created_at
        description: Event created at timestamp
      - name: ip_address        
        description: Event IP address
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
      - name: product_id
        description: Product ID associated with the inventory
      - name: created_at
        description: Inventory created at timestamp 
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored         

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
      - name: user_id
        description: User ID associated with the order
      - name: status
        description: Order status 
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: ____ # Fill in the table name for staging products 
    description: Product information such as SKU and category
    columns: 
      - name: ____ # Fill in the column name for the ID
        description: ____ # Fill in a column description for the ID 
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
    

SELECT 
  id,
  name,
  latitude,
  longitude
FROM {{ ref('looker__distribution_centers') }}

SELECT
  id,
  user_id,
  sequence_number,
  session_id,
  created_at,
  ip_address,
  city,
  state,
  postal_code,
  browser,
  traffic_source,
  uri,
  event_type
FROM {{ source('looker_ecommerce', 'events') }}

SELECT
  id,
  product_id,
  created_at,
  sold_at,
  cost,
  product_category,
  product_name,
  product_brand,
  product_retail_price,
  product_department,
  product_sku,
  product_distribution_center_id
FROM {{ source('looker_ecommerce', 'inventory_items') }}

SELECT
  id,
  order_id,
  user_id,
  product_id,
  inventory_item_id,
  status,
  created_at,
  shipped_at,
  delivered_at,
  returned_at,
  sale_price
FROM {{ source('looker_ecommerce', 'order_items') }}

SELECT 
  order_id,
  user_id,
  status,
  gender,
  created_at,
  returned_at,
  shipped_at,
  delivered_at,
  num_of_item
FROM {{ source('looker_ecommerce', 'orders') }}

SELECT
  id,
  cost,
  category,
  name,
  brand,
  retail_price,
  department,
  sku,
  distribution_center_id
FROM {{ source('looker_ecommerce', 'products') }}

SELECT 
  id,
  first_name,
  last_name,
  email,
  age,
  gender,
  state,
  street_address,
  postal_code,
  city,
  country,
  latitude,
  longitude,
  traffic_source,
  created_at
FROM {{ source('looker_ecommerce', 'users') }}

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
      - name: name 
        description: Distribution center name
      - name: latitude
        description: Distribution center location latitude
      - name: longitude # Fill in the column name for longitude
        description: Distribution center location longitude # Fill in the column description

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
      - name: first_name
        description: User first name
      - name: last_name
        description: User last name
      - name: email
        description: User email
      - name: age
        description: User age in years
      - name: gender
        description: User gender
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
      - name: created_at
        description: User created at timestamp

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
      - name: created_at
        description: Event created at timestamp
      - name: ip_address        
        description: Event IP address
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
      - name: product_id
        description: Product ID associated with the inventory
      - name: created_at
        description: Inventory created at timestamp 
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored         

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
      - name: user_id
        description: User ID associated with the order
      - name: status
        description: Order status 
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products # Fill in the table name for staging products 
    description: Product information such as SKU and category
    columns: 
      - name: id # Fill in the column name for the ID
        description: Unique ID for the product # Fill in a column description for the ID 
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
    

Adicionar descrições de colunas no dbt

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
        data_tests:
          - ____ # Fill in the data test for uniqueness
          - ____ # Fill in the data test for not null          
      - name: name 
        description: Distribution center name
        data_tests:
          - ____ # Fill in the data test for not null
      - name: latitude
        description: Distribution center location latitude
      - name: longtitude
        description: Distribution center location longitude

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null     
      - name: first_name
        description: User first name
        data_tests:
          - not_null
      - name: last_name
        description: User last name
        data_tests:
          - not_null
      - name: email
        description: User email
        data_tests:
          - not_null
      - name: age
        description: User age in years
        data_tests:
          - not_null       
      - name: gender
        description: User gender
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
        data_tests:
          - ____ # Fill in the data test for not null
          - ____ # Fill in the data test for an accepted_values test
              values: ____ # Fill in values accepted for an accepted_values test
      - name: created_at
        description: User created at timestamp
        data_tests:
          - not_null

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
        data_tests:
          - not_null
      - name: created_at
        description: Event created at timestamp
        data_tests:
          - not_null
      - name: ip_address        
        description: Event IP address
        data_tests:
          - not_null
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
        data_tests:
          - not_null
          - accepted_values:
              values: ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube']
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type        

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
        data_tests:
          - unique
          - not_null
      - name: product_id
        description: Product ID associated with the inventory
        data_tests:
          - not_null
      - name: created_at
        description: Inventory created at timestamp 
        data_tests:
          - not_null
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored 

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
        data_tests:
          - unique
          - not_null
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID associated with the order
        data_tests:
          - not_null        
      - name: status
        description: Order status 
        data_tests:
          - not_null        
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products
    description: Product information such as SKU and category
    columns: 
      - name: id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
      - name: distribution_center_id
        description: Distribution center ID where the product is stored 


version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
        data_tests:
          - unique # Fill in the data test for uniqueness
          - not_null # Fill in the data test for not null
      - name: name 
        description: Distribution center name
        data_tests:
          - not_null # Fill in the data test for not null
      - name: latitude
        description: Distribution center location latitude
      - name: longtitude
        description: Distribution center location longitude

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
      - name: first_name
        description: User first name
        data_tests:
          - not_null
      - name: last_name
        description: User last name
        data_tests:
          - not_null
      - name: email
        description: User email
        data_tests:
          - not_null
      - name: age
        description: User age in years
        data_tests:
          - not_null 
      - name: gender
        description: User gender
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
        data_tests:        
          - not_null # Fill in the data test for not null
          - accepted_values: # Fill in the data test for an accepted_values test
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search'] # Fill in values accepted for an accepted_values test
      - name: created_at
        description: User created at timestamp
        data_tests:
          - not_null

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
        data_tests:
          - not_null
      - name: created_at
        description: Event created at timestamp
        data_tests:
          - not_null
      - name: ip_address        
        description: Event IP address
        data_tests:
          - not_null
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
        data_tests:
          - not_null
          - accepted_values:
              values: ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube']
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
        data_tests:
          - unique
          - not_null
      - name: product_id
        description: Product ID associated with the inventory
        data_tests:
          - not_null
      - name: created_at
        description: Inventory created at timestamp 
        data_tests:
          - not_null
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored 

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
        data_tests:
          - unique
          - not_null
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID associated with the order
        data_tests:
          - not_null        
      - name: status
        description: Order status 
        data_tests:
          - not_null        
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products
    description: Product information such as SKU and category
    columns: 
      - name: id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
      - name: distribution_center_id
        description: Distribution center ID where the product is stored 


Adicionar testes de coluna do dbt

Construindo data marts e modelos de snapshot no dbt

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
        data_tests:
          - unique
          - not_null          
      - name: name 
        description: Distribution center name
        data_tests:
          - not_null
      - name: latitude
        description: Distribution center location latitude
      - name: longtitude
        description: Distribution center location longitude

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
      - name: first_name
        description: User first name
        data_tests:
          - not_null
      - name: last_name
        description: User last name
        data_tests:
          - not_null
      - name: email
        description: User email
        data_tests:
          - not_null
      - name: age
        description: User age in years
        data_tests:
          - not_null 
      - name: gender
        description: User gender
        data_tests:
          - accepted_values:
              values: ['F', 'M', null] 
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
      - name: created_at
        description: User created at timestamp
        data_tests:
          - not_null

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
        data_tests:
          - not_null
      - name: created_at
        description: Event created at timestamp
        data_tests:
          - not_null
      - name: ip_address        
        description: Event IP address
        data_tests:
          - not_null
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
        data_tests:
          - unique
          - not_null
      - name: product_id
        description: Product ID associated with the inventory
        data_tests:
          - not_null
      - name: created_at
        description: Inventory created at timestamp 
        data_tests:
          - not_null
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored 

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
        data_tests:
          - unique
          - not_null
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID associated with the order
      - name: status
        description: Order status 
      - name: gender
        description: Gender of the user associated with the order
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products
    description: Product information such as SKU and category
    columns: 
      - name: id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
      - name: distribution_center_id
        description: Distribution center ID where the product is stored 


{% snapshot inventory_snapshot %}

{{
    config(
      target_schema='main',
      unique_key='id',
      strategy='timestamp',
      updated_at='created_at' 
    )
}}

SELECT * 
FROM {{ source('looker_ecommerce', 'inventory_items') }}

{% endsnapshot %}


{% snapshot orders_snapshot %}

{{
    config(
      target_schema='main',
      unique_key='order_id',
      strategy='timestamp',
      updated_at='created_at' 
    )
}}

SELECT * 
FROM {{ source('looker_ecommerce', 'orders') }}

{% endsnapshot %}


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users')}}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    -- dimensions
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    -- facts 
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at is NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    -- WHERE sold_at IS NULL
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,
    -- dimensions
    pb.product_name,
    pb.product_category,
    -- facts (e.g. profit analysis)
    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit
FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id

Relembrando a hierarquia de modelos do dbt

version: 2

models:
  - name: stg_looker__distribution_centers
    description: Distribution center name and location
    columns:
      - name: id 
        description: Unique ID for the distribution center
        data_tests:
          - unique
          - not_null          
      - name: name 
        description: Distribution center name
        data_tests:
          - not_null
      - name: latitude
        description: Distribution center location latitude
      - name: longtitude
        description: Distribution center location longitude

  - name: stg_looker__users
    description: Users demographics and location information
    columns:
      - name: id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
      - name: first_name
        description: User first name
        data_tests:
          - not_null
      - name: last_name
        description: User last name
        data_tests:
          - not_null
      - name: email
        description: User email
        data_tests:
          - not_null
      - name: age
        description: User age in years
        data_tests:
          - not_null 
      - name: gender
        description: User gender
      - name: state
        description: User residing state
      - name: street_address
        description: User residing street address
      - name: postal_code
        description: User residing postal code
      - name: city
        description: User residing city 
      - name: country
        description: User residing country
      - name: latitude
        description: User residing address latitude
      - name: longitude
        description: User residing address longitude
      - name: traffic_source
        description: Traffic source for user information 
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']        
      - name: created_at
        description: User created at timestamp
        data_tests:
          - not_null

  - name: stg_looker__events
    description: User event data
    columns: 
      - name: id
        description: Unique ID for event entity
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID tied to the event 
      - name: sequence_number
        description: Sequence number for the event 
      - name: session_id
        description: Event session ID 
        data_tests:
          - not_null
      - name: created_at
        description: Event created at timestamp
        data_tests:
          - not_null
      - name: ip_address        
        description: Event IP address
        data_tests:
          - not_null
      - name: city
        description: City where event occurred 
      - name: state
        description: State where event occurred
      - name: postal_code
        description: Postal code where event occurred
      - name: browser
        description: Browser used for event 
      - name: traffic_source
        description: Traffic source for event information
        data_tests:
          - not_null
          - accepted_values:
              values: ['Adwords', 'Email', 'Facebook', 'Organic', 'YouTube']           
      - name: uri
        description: Event URI
      - name: event_type    
        description: Event type

  - name: stg_looker__inventory_items
    description: Inventory items information
    columns: 
      - name: id
        description: Unique ID for the inventory item
        data_tests:
          - unique
          - not_null
      - name: product_id
        description: Product ID associated with the inventory
        data_tests:
          - not_null
      - name: created_at
        description: Inventory created at timestamp 
        data_tests:
          - not_null
      - name: sold_at
        description: Timestamp inventory item was sold at 
      - name: cost
        description: Cost of the item 
      - name: product_category
        description: Product category
      - name: product_name
        description: Product name
      - name: product_brand
        description: Product brand 
      - name: product_retail_price
        description: Product retail price
      - name: product_department
        description: Product department
      - name: product_sku
        description: Product SKU number
      - name: product_distribution_center_id
        description: Distribution center ID where the product is stored 

  - name: stg_looker__order_items
    description: Order information on the item level
    columns: 
      - name: id
        description: Unique ID for the order item
        data_tests:
          - unique
          - not_null
      - name: order_id
        description: Order ID associated with the order item
      - name: user_id
        description: User ID associated with the order item
      - name: product_id
        description: Product ID associatd with the order item 
      - name: inventory_item_id
        description: Inventory item ID associated with the order item 
      - name: status
        description: Status of the order
      - name: created_at
        description: Order item created at timestamp 
      - name: shipped_at
        description: Order item shipped at timestamp
      - name: delivered_at
        description: Order item delivered at timestamp
      - name: returned_at
        description: Order item returned at timestamp
      - name: sale_price
        description: Order item sale price

  - name: stg_looker__orders
    description: Order information such as status and timestamps
    columns:
      - name: order_id
        description: Unique ID for the order
        data_tests:
          - unique
          - not_null
      - name: user_id
        description: User ID associated with the order
        data_tests:
          - not_null        
      - name: status
        description: Order status 
        data_tests:
          - not_null        
      - name: gender
        description: Gender of the user associated with the order
        data_tests:
          - not_null     
      - name: created_at
        description: Order created at timestamp
      - name: returned_at
        description: Order returned at timestamp 
      - name: shipped_at
        description: Order shipped at timestamp
      - name: delivered_at
        description: Order delivered at timestamp
      - name: num_of_item    
        description: Number of items ordered
    
  - name: stg_looker__products
    description: Product information such as SKU and category
    columns: 
      - name: id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
      - name: cost
        description: Product cost
      - name: category
        description: Product category
      - name: name
        description: Product name 
      - name: brand
        description: Product brand 
      - name: retail_price
        description: Product retail price
      - name: department
        description: Product department
      - name: sku
        description: Product SKU number
      - name: distribution_center_id
        description: Distribution center ID where the product is stored 


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          ____ # Fill in uniqueness test
          ____ # Fill in not null test 
          ____ # Fill in relationship test to staging users
            ____
            ____
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null 
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null 
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests: 
          - not_null
          ____ # Fill in accepted value test 
            ____
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM stg_looker__users _____ -- Replace with dbt ref model
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM stg_looker__order_items _____ -- Replace with dbt ref model
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM stg_looker__orders _____ -- Replace with dbt ref model
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM stg_looker__events _____ -- Replace with dbt ref model
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users')}}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

Criar modelo de mart para customers

version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: _____ # Fill in table name 
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
		____ # Declare the data_tests for product ID
          ____ # Fill in unique, not null, and foreign key data tests
          ____
          ____
            ____
            ____
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
		____ # Fill in not null datatest for product category
          ____
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold
        

WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
	-- Fill in the SQL syntax for creating a CTE with two columns: product_id, and a calculated sales amount per product
    SELECT 
        product_id,
        ____ AS sales_amount
    FROM ____
    GROUP BY ____
)

SELECT 
    pb.product_id,

    pb.product_name,
    pb.product_category,

    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id

WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
    -- Fill in the SQL syntax for creating a CTE with two columns: product_id, and a calculated sales amount per product
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,

    pb.product_name,
    pb.product_category,

    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


Criar modelo de mart para produtos

WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref(stg_looker__users) }}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,

    pb.product_name,
    pb.product_category,

    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_items ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

- name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


WITH customer_base AS (
    SELECT 
        id AS user_id,
        first_name AS customer_first_name,
        last_name AS customer_last_name,
        country AS customer_country,
        traffic_source AS customer_acquisition_channel
    FROM {{ ref('stg_looker__users') }}
)

, order_items AS (
    SELECT 
        user_id,
        SUM(sale_price) AS total_amount_spent,
        COUNT(DISTINCT id) AS total_items_purchased,
        MIN(created_at) AS first_order_completed_at,
        MAX(created_at) AS last_order_completed_at,
    FROM {{ ref('stg_looker__order_items') }}
    WHERE status = 'Complete'
    GROUP BY 1
)

, orders AS (
    SELECT 
        user_id,
        COUNT(DISTINCT order_id) AS num_orders,
        COUNT(DISTINCT 
            CASE WHEN status = 'Shipped'
            THEN order_id
            END) AS num_orders_shipped,
        COUNT(DISTINCT 
            CASE WHEN status = 'Complete'
            THEN order_id
            END) AS num_orders_complete,
        COUNT(DISTINCT 
            CASE WHEN status = 'Processing'
            THEN order_id
            END) AS num_orders_processing,
        COUNT(DISTINCT 
            CASE WHEN status = 'Cancelled'
            THEN order_id
            END) AS num_orders_cancelled,
        COUNT(DISTINCT 
            CASE WHEN status = 'Returned'
            THEN order_id
            END) AS num_orders_returned,
    FROM {{ ref('stg_looker__orders') }}
    GROUP BY 1
)

, web_traffic AS (
    SELECT 
        user_id,
        COUNT(DISTINCT session_id) AS num_web_sessions,
    FROM {{ ref('stg_looker__events') }}
    GROUP BY 1
)

SELECT
    cb.user_id,
    cb.customer_first_name,
    cb.customer_last_name,
    cb.customer_country,
    cb.customer_acquisition_channel,
    oi.total_amount_spent,
    oi.total_items_purchased,
    oi.first_order_completed_at,
    oi.last_order_completed_at,
    o.num_orders,
    o.num_orders_shipped,
    o.num_orders_complete,
    o.num_orders_processing,
    o.num_orders_cancelled,
    o.num_orders_returned,
    wt.num_web_sessions,
FROM customer_base cb
LEFT JOIN order_items oi 
    on cb.user_id = oi.user_id
LEFT JOIN orders o
    ON cb.user_id = o.user_id 
LEFT JOIN web_traffic wt 
    ON cb.user_id = wt.user_id


Depurando erros comuns

WITH product_base AS (
    SELECT 
        id AS product_id,
        name AS product_name,
        category AS product_category,
        department AS product_department
    FROM {{ ref('stg_looker__products') }}
)

, inventory_items AS (
    SELECT 
        product_id,
        SUM(CASE WHEN sold_at IS NOT NULL THEN cost END) AS cost_of_goods_sold
    FROM {{ ref('stg_looker__inventory_items') }}
    GROUP BY 1
)

, order_items AS (
    SELECT 
        product_id,
        SUM(sale_price) AS sales_amount
    FROM {{ ref('stg_looker__order_items') }}
    GROUP BY 1    
)

SELECT 
    pb.product_id,

    pb.product_name,
    pb.product_category,

    oi.sales_amount,
    ii.cost_of_goods_sold,
    oi.sales_amount - ii.cost_of_goods_sold AS profit

FROM product_base pb 
LEFT JOIN inventory_item ii 
    ON pb.product_id = ii.product_id 
LEFT JOIN order_items oi 
    ON pb.product_id = oi.product_id


version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: user_id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', 'Search']
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


Depurando mais erros

version: 2

models:
  - name: customers
    description: Data mart for customer-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: user_id
        description: Unique ID for the user
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__users')
              field: id
      - name: customer_first_name
        description: Customer first name
        data_tests:
          - not_null
      - name: customer_last_name
        description: Customer last name
        data_tests:
          - not_null
      - name: customer_country
        description: Customer country
      - name: customer_acquisition_channel
        description: Customer acquisition channel
        data_tests:
          - not_null
          - accepted_values:
              values: ['Display', 'Email', 'Facebook', 'Organic', Search]
      - name: total_amount_spent 
        description: Total amount customer has spent
      - name: total_items_purchased
        description: Total items purchased
      - name: first_order_completed_at
        description: When the first order was completed at, if at all
      - name: last_order_completed_at
        description: When the last order was completed at, if at all
      - name: num_orders
        description: Total number of orders placed
      - name: num_orders_shipped
        description: Number of orders shipped
      - name: num_orders_complete
        description: Number of orders completed
      - name: num_orders_processing
        description: Number of orders still processing
      - name: num_orders_cancelled
        description: Number of orders cancelled
      - name: num_orders_returned
        description: Number of orders returned
      - name: num_web_sessions
        description: Number of web sessions

  - name: products
    description: Data mart for product-related dimensions and facts, for analytical and reporting purposes
    columns:
      - name: product_id
        description: Unique ID for the product
        data_tests:
          - unique
          - not_null
          - relationships:
              to: ref('stg_looker__products')
              field: id
      - name: product_name
        description: Product name
      - name: product_category
        description: Product category
        data_tests:
          - not_null
      - name: sales_amount
        description: Total sales amount attributed to this product
      - name: cost_of_goods_sold
        description: Total cost of goods sold attributed to this product
      - name: profit
        description: Profit, calculated by sales amount minus cost of goods sold


Criando modelos de snapshot no dbt

Construindo modelos no dbt

Aprenda a aprimorar seus projetos dbt com Jinja, simplificando o código, reduzindo redundâncias e melhorando a manutenibilidade. Você vai praticar o uso de variáveis, loops e macros para construir fluxos de transformação mais eficientes.

Introdução ao Jinja com dbt

Aquecendo com dbt compile

Aquecendo com dbt compile 2

Usando dbt compile para depurar

Definindo variáveis com dbt Jinja

Definindo variáveis com dbt Jinja 2

Uso avançado de Jinja com dbt

Usando loops for do Jinja em SQL repetitivo 1

Usando loops do Jinja em SQL repetitivo 2

Macros com Jinja

Removendo espaços em branco com Jinja

Recapitulação: estudo de caso com dbt

Aprimorando o dbt com Jinja

Desbloqueie o poder da transformação de dados com nosso curso prático de estudo de caso em dbt (Data Build Tool). Projetado para quem está começando com dbt, este curso foca em exercícios práticos e reais que oferecem várias oportunidades para você criar, transformar e consolidar seu domínio dos comandos essenciais do dbt. Com prática guiada, você vai ganhar confiança em cada etapa do processo — desde a configuração do ambiente dbt até a criação de modelos avançados que resolvem desafios críticos de negócio. Ao realizar exercícios interativos, você desenvolverá a experiência necessária para encarar projetos reais de pipelines ETL de dados com segurança.

Transforme dados brutos em modelos limpos e confiáveis usando o dbt (Data Build Tool) — uma estrutura de transformação moderna baseada em SQL usada por equipes de dados em todo o mundo. Esse curso prático de estudo de caso foi feito pra quem tá começando e quer aprender habilidades reais com exercícios práticos e guiados. Você vai configurar seu próprio ambiente dbt, modelar dados em escala e escrever código reutilizável usando os recursos integrados do dbt.<br><br><h2>Configure seu projeto e explore os dados</h2>Comece configurando um projeto dbt e trabalhando com um conjunto de dados real de comércio eletrônico. Você vai organizar dados brutos, configurar perfis e resolver problemas de sintaxe enquanto aprende sobre o contexto de negócios por trás de cada etapa da transformação.<br><br><h2>Criar e validar modelos</h2>Aprenda a criar modelos de preparação escaláveis e aplique verificações de qualidade de dados para garantir que seus conjuntos de dados sejam precisos e estejam prontos para análise. Você vai criar uma base sólida para responder às principais questões de negócios.<br><br><h2>Automatize com Jinja</h2>Conclua o curso aprendendo a usar o Jinja para escrever código reutilizável e fácil de manter. Você vai usar variáveis, fluxo de controle e loops para seguir o princípio DRY (Don't Repeat Yourself, ou Não se repita) para otimizar seu fluxo de trabalho dbt.

Data Manipulation in SQL

Intermediate dbt

Aprenda a transformar dados brutos em modelos confiáveis com dbt através de exercícios práticos.

Estudo de caso: criando modelos de dados de e-commerce com dbt

Aprenda a transformar dados brutos em modelos limpos e confiáveis com o dbt por meio de exercícios práticos e reais. 

Construindo data marts e modelos de snapshot no dbt

Create Your Free Account