This module offers an overview of the course and outlines the learning objectives.

Course introduction

Course overview

This module details the role of storage infrastructure in the AI data pipeline. It covers performance demands, key Google Cloud solutions, and the decision criteria for selecting a service based on capacity, throughput, and latency.

AI data pipeline and storage solutions

Key storage metrics

Quiz Question 1

Foundations of AI storage

This module details the critical phases of data preparation and model training within the AI workflow. It covers optimizing data loading using Cloud Storage, Anywhere Cache, and the Dataflux Dataset tool, while comparing high-performance file systems like Cloud Storage FUSE and Managed Lustre. Additionally, it outlines decision criteria for efficient checkpointing strategies to ensure fault tolerance and minimize GPU idle time.

Data preparation

Data loading

Checkpointing

Cloud Storage FUSE

Managed Lustre

Quiz Question 2

Quiz Question 3

Prepare and train

This module details strategies for AI model serving and data archiving. It covers selecting storage—Managed Lustre, Cloud Storage, or Hyperdisk ML—based on scale and latency, and optimization techniques, like GKE Image Streaming and Cloud Storage FUSE, to minimize costs and load times.

Model serving or inference

Accelerating inference with GKE Image Streaming

Optimizing data retrieval with Cloud Storage Fuse and Hyperdisk ML

Data archiving

Reference architectures

Congratulations and next steps

Quiz Question 4

Serve and archive

Course resources

In this course, you’ll take a comprehensive journey through the storage solutions available on Google Cloud, specifically tailored for AI and high-performance computing (HPC) workloads. You’ll learn how to choose the right storage for each stage of the ML lifecycle. You’ll explore how to optimize for I/O performance during training, manage massive datasets for data preparation, and serve model artifacts with low latency. Through practical examples and demonstrations, you’ll gain the expertise to design robust storage solutions that accelerate your AI innovation.

AI Infrastructure Storage Options

Journey through the storage solutions available on Google Cloud, specifically tailored for AI and high-performance computing (HPC) workloads. 

AI Infrastructure: Storage Options

Optimizing data retrieval with Cloud Storage Fuse and Hyperdisk ML

Interaktive praktische Übung