This chapter will teach you the basics of Dask and lazy evaluation. At the end of this chapter, you'll be able to speed up almost any Python code by using parallel processing or multi-threading. You'll learn the difference between these two task scheduling methods and which one is better under which circumstances.

Introduction to Dask

Lazy evaluation

Delaying functions

Task graphs and scheduling methods

Options

Multi-threading

Parallel processing

What are the different schedulers?

Plotting the task graph

Building delayed pipelines

Analyzing songs on Spotify

How danceable are songs these days?

Most popular songs

 Lazy Evaluation and Parallel Computing

Here you’ll learn how to analyze big structured data using Dask arrays and Dask DataFrames. You'll learn how everything you know about NumPy and pandas can easily be applied to data that is too large to fit in memory.

Dask arrays

Dask array chunksizes

Loading and processing photos

An image processing pipeline

Dask DataFrames

Creating Dask dataframes from CSVs

Read Dask DataFrames from Parquet

Summertime grooves

Multidimensional arrays

Exploring HDF5 files

Dask arrays from HDF5 datasets

Dask arrays from Zarr datasets

Xarray

Exploratory data analysis with xarray

Monthly mean temperatures

Calculating the trend in European temperatures

Parallel Processing of Big, Structured Data

Process any kind of data. You'll learn how Dask bags can be used to efficiently process unstructured text data, semi-structured JSON data, and even recorded audio.

Introduction to Dask bags

Creating a Dask bag

Creating a bag from saved text

String operations

Dask bag operations

Loading JSON data

Filtering Dask bags

Chaining operations

Converting unstructured data to a DataFrame

Restructuring a dictionary

Converting to DataFrame

Using any data in Dask bags

Loading wav data

Constructing custom Dask bags

Processing unstructured audio

Dask Bags for Unstructured Data

Harness the power of Dask to train machine learning models. You'll learn how to train machine learning models on big data using the Dask-ML package, and how to split Dask calculations across a mixture of processes and threads for even greater computing speed.

Using processes and threads

Which scheduler will be used?

Clusters and clients

Training machine learning models on big datasets

Using Dask to train a linear model

Making lazy predictions

Preprocessing big datasets

Lazily transforming training data

Lazy train-test split

Wrap-up

Dask Machine Learning and Final Pieces

Spotify Songs - CSV

Spotify Songs - Parquet

European Rainfall - HDF5

European Rainfall - Zarr

Tripadvisor Hotel Reviews

Politicians

When working with big data, you’ll face two common obstacles: using too much memory and long runtimes. The Dask library can lower your memory use by loading chunks of data only when needed. It can lower runtimes by using all your available computing cores in parallel. Best of all, it requires very few changes to your existing Python code. In this course, you use Dask to analyze Spotify song data, process images of sign language gestures, calculate trends in weather data, analyze audio recordings, and train machine learning models on big data.

<h2>Use Parallel Processing to Speed Up Your Python Code</h2>
With this 4-hour course, you’ll discover how parallel processing with Dask in Python can make your workflows faster. 
<br><br>
When working with big data, you’ll face two common obstacles: using too much memory and long runtimes. The Dask library can lower your memory use by loading chunks of data only when needed. It can lower runtimes by using all your available computing cores in parallel. Best of all, it requires very few changes to your existing Python code.
<br><br>
<h2>Analyze Big Structured Data Using Dask DataFrames</h2>
In this course, you use Dask to analyze Spotify song data, process images of sign language gestures, calculate trends in weather data, analyze audio recordings, and train machine learning models on big data.
<br><br>
You’ll start by learning the basics of Dask, exploring how parallel processing in Python can speed up almost any code. Next, you’ll explore Dask DataFrames and arrays and how to use them to analyze big structured data. 
<br><br>
<h2>Train machine learning models using Dask-ML</h2>
As you progress through the 51 exercises in this course, you’ll learn how to process any type of data, using Dask bags to work with unstructured and structured data. Finally, you’ll learn how to use Dask in Python to train machine learning models and improve your computing speeds.

Data Manipulation with pandas

Python Toolbox

Learn how to use Python parallel programming with Dask to upscale your workflows and efficiently handle big data. 

Parallel Programming with Dask in Python

Climate Informatics Researcher

Speeding up computations using multiple cores

Concurrent programming

Parallel programming

Dask delayed

Computing the answer

Using operations on delayed objects

Sharing computation

Let's practice!

https://s3.amazonaws.com/assets.datacamp.com/production/course_27055/subtitles/course_27055_c4c4964b9ce77ec6b2e7238138203ec7.vtt

 Lazy Evaluation and Parallel Computing - Introduction to Dask

Introduction to Dask

Create Your Free Account