Learn how to connect to live data sources and prepare time series data for forecasting. You’ll pull hourly electricity demand data from the U.S. EIA API and build your first forecast.

Productionizing your forecast model

Forecasting project architecture

Benefits of experimentation frameworks

Reviewing the input data

Making an API request

Preparing and visualizing the data

Interpreting time series visualizations

Working with a forecast object

Forecasting with ML Models

Evaluating forecast performance

Visualizing forecast performance

General Architecture

Discover the fundamentals of experimentation, including backtesting, evaluation, and model registration using MLflow!

Experimentation

Forecast evaluation & experimentation

Exploring experimentation process

Workflow of experimentation

Backtesting

Defining the forecasting models

Training models with backtesting

Visualizing forecast results

Model Scoring

Transforming backtesting output

Building a model leaderboard

Model Registration with MLflow

Logging tuned models

Searching experiment results

Learn how to build automated forecasting pipelines that refresh data and predictions daily. You'll set up ETL processes, register models with MLflow, and orchestrate everything with Airflow. Create a production-ready system with data validation and logging to monitor pipeline health.

Pipeline architecture

Defining the forecasting pipeline

Fitting the model

Registering the model

Data ingestion

Data parity checks

Defining the schema

Performing data validation

Airflow

Setting up the DAG

Creating the DAG

Defining pipeline execution order

Logs

Monitoring ETL logs

Monitoring forecasting logs

Setting Automation

Discover the essentials of production deployment, from monitoring the pipeline health to detecting model drift. You'll learn best practices for reproducibility, scaling, and maintaining forecasting systems in real-world environments.

Pipeline Deployment to Production

Options

Data Integrity Issues

Operational Failures

Infrastructure Failures

What could go wrong?

Observability

Monitoring

Monitoring vs Observability

Pipeline Monitoring

Data Integrity

Availability

Restatements

Categorizing API risks

Building a mitigation plan

Model Drift

Calculating a threshold and moving averages

Identifying model drift

Best practices

Development best practices

Scaling forecasting pipelines

Congratulations!

From Deployment to Production

Learn how to design, automate, and monitor scalable forecasting pipelines in Python. This advanced course walks you through the entire production workflow - from sourcing data and training models to deployment and monitoring - using tools like MLflow and Airflow.

You'll start by connecting to live data sources and building your first forecast with U.S. electricity demand data. Next, you'll discover experimentation fundamentals, including backtesting, evaluation, and model registration using MLflow.

Then you'll build automated forecasting pipelines with ETL processes, model registration, and Airflow orchestration. Finally, you'll learn production deployment essentials, including monitoring pipeline health, detecting model drift, and maintaining forecasting systems in real-world environments.

Introduction to Apache Airflow in Python

Introduction to MLflow

Time Series Analysis in Python

Learn how to design, automate, and monitor scalable forecasting pipelines in Python. 

Designing Forecasting Pipelines for Production

The Weather API changed temperature units from Celsius to Fahrenheit without notice.

Solar panel sensor readings stopped arriving after the maintenance window.

Pipeline crashed due to an incompatible pandas version after the update.

Forecast calculation failed with "division by zero" error.

Pipeline timeout after cloud provider network outage lasted 3 hours.

GPU memory allocation failed during model training on the shared cluster.

What could go wrong?

Designing Forecasting Pipelines for Production

Praktische interactieve oefening