1. Data warehouses support organizational analysis
This video looks at the data warehouse life cycle and the different personas that support the different stages. In this video, we will use hypothetical characters as personas and talk about the life-cycle stages in general terms. However, others might organize and categorize the stages differently.
2. High-level life cycle
To start with, let's talk about the life cycle of a data warehouse project. At a high level, the first step is the planning phase, where the team begins to plan how to design the data warehouse to satisfy the organization's needs. Next is the implementation phase, where the team builds the data warehouse. Finally, in the support and maintenance phase, the team trains end users and maintain the warehouse.
3. Planning - business requirements
In the planning phase, there are different sub-stages. The first is requirements gathering. The goal here is to understand the organization's needs. Who and how will they use the data warehouse? To support this task, meet Christina, a data analyst, and Alex, a Data Scientist. Christina works closely with others in the business, and her analysis helps with decision-making. Alex's ML models automate decision-making and improve business processes. Christina and Alex are closer to the business and know their final goals and requirements best.
4. Planning - data modeling
Data modeling is next. It is planning based on business requirements on how the team transforms data from different input sources and integrates it into our data warehouse. Crucial is that the team understands and links the relevant data sets.
To support this stage, meet Stacy and Derrick. Stacy is a data engineer and is skilled at creating data pipelines. Data pipelines are automatic end-to-end processes that collect, modify, and deliver data. Derrick is a transactional database admin. Data engineers and admins plan data pipelines from the database systems to the warehouse.
Also, Christina and Alex use their business knowledge to support this stage by helping ensure the data model accurately represents the organization.
5. Implementation - ETL Design & Development
Within the implementation phase, the team designs the ETL process. This step is about designing and building the data pipelines that extract, transform, and load data from the different sources into the data warehouse.
Stacy is responsible for creating the pipelines, but she coordinates with Derrick to extract data from the source systems.
6. Implementation - BI Application Development
After the team loads the data into the warehouse, they work on BI application development. In this step, they set up BI or business intelligence tools to interact with the data warehouse and create reports needed by the organization.
BI tools are often how many users interact with the data warehouse. Some standard BI tools include Tableau, Power BI, or Google's Looker. At this point, Christina and Alex consult on the setup of the BI application.
7. Support / Maintenance - Maintenance
During the support and maintenance phase, the team can update the warehouse table designs or make other necessary changes. Stacy makes these changes if required.
8. Support / Maintenance - Test & Deploy
After this step, we get to testing and deployment. First, Christina and Alex test the system to confirm their business requirements are met. Afterward, Stacy will deploy and make the warehouse available to the organization.
After deployment, any significant changes will follow the same steps starting back at the planning phase.
9. Persona matrix
In this video, we have covered a few personas that support the data warehouse life cycle. This chart summarizes when the different personas are needed. The list contains only the leading data-centric roles involved. For example, we ignored roles like project managers.
Finally, additional essential personas are the organizational leaders who sponsor the implementation and remove administrative barriers to completing the project.
10. Let's practice!
Okay, time for practice!