Merancang pipeline DVC

Merancang pipeline DVC, atau DAG, merupakan dasar untuk memanfaatkan DVC dalam alur kerja Machine Learning Anda. DAG memungkinkan kita mengodekan input, output, dan eksekusi suatu langkah. Output dari satu langkah dapat menjadi input bagi satu atau lebih langkah lain, sehingga secara alami menetapkan dependensi yang tepat antarlangkah.

Dalam latihan ini, Anda akan merancang alur kerja ML yang berisi empat tahap, yaitu,

Prapemrosesan data (preprocess_stage)
Pemisahan data (split_stage)
Pelatihan model (train_stage)
Evaluasi model (evaluate_stage)

Kita hanya akan menggunakan perintah dvc stage add. Gulir ke bagian akhir berkas shell script (dvc_dag_stages_add.sh) jika diperlukan.

Latihan ini adalah bagian dari kursus

Pengantar Versioning Data dengan DVC

Lihat Kursus

Latihan interaktif praktis

Ubah teori menjadi tindakan dengan salah satu latihan interaktif kami.

Mulai berolahraga

Latihan ini adalah bagian dari kursus

Pengantar Versioning Data dengan DVC

SkillTag.level.intermediateSkillTag.label

4.8+

323 reviews

Mulai Kursus Gratis

This chapter provides a comprehensive introduction to Data Version Control (DVC), a tool essential for data versioning in machine learning. Learners will explore the motivation behind data versioning, understand its differences from code versioning, and experiment with a simple classification problem. They will review basic Git commands, learn about DVC, and practice setting up a repository. The chapter concludes with an overview of DVC’s features and use cases, including versioning data and models, CI/CD for machine learning, experiment tracking, pipelines, and more.

Exercise 1: Data Versioning Motivation Exercise 2: Anatomy of a Machine Learning Model Exercise 3: Differences Between Data and Code Versioning Exercise 4: Understanding Hyperparameters Exercise 5: Introduction to DVC Exercise 6: Working with Git CLI Exercise 7: Review DVC CLI Exercise 8: DVC features and use cases Exercise 9: DVC pipelines Exercise 10: CI/CD for machine learning

This chapter delves into the setup of DVC, encompassing aspects such as installation, initialization of the repository, and the utilization of the .dvcignore file. It further navigates through the exploration of DVC cache and staging files, imparting knowledge on how to add and remove files, manage caches, and comprehend the underlying mechanisms using the MD5 hash. The chapter also elucidates on DVC remotes, distinguishing them from Git remotes, and guides you on how to add, list, and modify them. Lastly, it teaches you how to interact with these remotes by pushing and pulling data, checking out specific versions, and fetching data to the cache.

Exercise 1: DVC Setup and Initialization Exercise 2: Setting up DVC Exercise 3: .dvcignore Patterns Exercise 4: DVC Cache and Staging Files Exercise 5: Working with DVC Cache Exercise 6: Understanding .dvc Files Exercise 7: Configuring DVC Remotes Exercise 8: Purpose of DVC Remotes Exercise 9: Setup a DVC Remote Exercise 10: Interacting with DVC Remotes Exercise 11: Versioning Data using DVC Remote Exercise 12: Checking out Versioned Data

This chapter focuses on automating ML pipelines using DVC. Learners create a configuration file containing settings and hyperparameters. They also learn about pipeline visualization using directed acyclic graphs and use commands to describe dependencies, commands, and outputs. Execution of DVC pipelines is covered, including local model training and how Git tracks DVC metadata. Additionally, learners explore metrics and plots tracking in DVC, including how to print metrics, create plot files, and compare metrics and plots across different pipeline stages.

Exercise 1: Organisasi kode dan refaktorisasi Exercise 2: Memahami berkas parameter di DVC Exercise 3: Tulis sebuah berkas parameter Exercise 4: Menulis dan memvisualisasikan pipeline DVC Exercise 5: Merancang pipeline DVC

Latihan Saat Ini

Exercise 6: Memvisualisasikan pipeline DVC Exercise 7: Menjalankan pipeline DVC Exercise 8: Konsep eksekusi pipeline DVC Exercise 9: Menjalankan pipeline pelatihan model ML Exercise 10: Evaluasi: Metrik dan plot di DVC Exercise 11: Melacak Metrik DVC Exercise 12: Menambahkan plot ke dvc.yaml Exercise 13: Selamat!