Introduction to deep learning with PyTorch

1. Introduction to deep learning with PyTorch

Hi. I'm Jasmin, a Senior Data Science Content Developer at DataCamp, and I'll be your instructor for this course on deep learning with PyTorch.

2. Deep learning is everywhere!

Deep learning is everywhere; it powers many recent and exciting innovations, from language translation

3. Deep learning is everywhere!

to self-driving cars,

4. Deep learning is everywhere!

medical diagnostics,

5. Deep learning is everywhere!

and chatbots.

6. What is deep learning?

Deep learning is a subset of machine learning, where the fundamental model structure is a network of inputs, hidden layers, and outputs, as shown. A network can have one

7. What is deep learning?

or many hidden layers.

8. Deep learning networks

The original intuition behind deep learning was to create models inspired by how the human brain learns: through interconnected cells called

9. Deep learning networks

neurons. This is why we continue to call deep learning models "neural" networks. These layered model structures require far more data compared to other machine learning models in order to derive patterns. We are usually talking about at least hundreds of thousands of data points.

10. PyTorch: a deep learning framework

While there are several frameworks and packages out there for implementing deep learning algorithms, we'll focus on PyTorch, one of the most popular and well-maintained frameworks. PyTorch was originally developed by Meta AI as part of Facebook's AI Research lab before it moved under the Linux Foundation. It is designed to be intuitive and user-friendly, sharing many similarities with the Python library NumPy.

11. PyTorch tensors

We can import the PyTorch module by calling import torch. The fundamental data structure in PyTorch is a tensor, which is similar to an array or matrix. It can support many mathematical operations and forms a building block for our neural networks. Tensors can be created from Python lists or NumPy arrays using the torch.tensor() class. This class converts the data into a compatible format for deep learning.

12. Tensor attributes

We can call tensor.shape to display the shape of our newly created tensor and tensor.dtype() to display its data type, here, a 64-bit integer. Checking the shape and data type ensures tensors align correctly with our model and task, and can help us in case of debugging.

13. Getting started with tensor operations

PyTorch tensors can be added or subtracted, provided that their shapes are compatible. Two tensors are compatible if they have the same shape, meaning their dimensions align exactly. When shapes are incompatible, we get an error.

14. Element-wise multiplication

We can also perform element-wise multiplication, which involves multiplying each corresponding element from two arrays of the same shape, and many other operations, including matrix multiplication.

15. Matrix multiplication

Matrix multiplication is a way of combining two matrices to make a new one.

16. Matrix multiplication

Each value in the new matrix comes from multiplying a row from the first matrix with a column from the second matrix and summing the results. Behind the scenes, deep learning models perform countless operations like addition and multiplication to process data and learn patterns.

17. Let's practice!

Let's manipulate some tensors!

This exercise is part of the course

Introduction to Deep Learning with PyTorch

IntermediateSkill Level

4.8+

Start Course for Free

Self-driving cars, smartphones, search engines... Deep learning is now everywhere. Before you begin building complex models, you will become familiar with PyTorch, a deep learning framework. You will learn how to manipulate tensors, create PyTorch data structures, and build your first neural network in PyTorch with linear layers.

Exercise 1: Introduction to deep learning with PyTorch

Current Exercise

Exercise 2: Getting started with PyTorch tensors Exercise 3: Checking and adding tensors Exercise 4: Neural networks and layers Exercise 5: Linear layer network Exercise 6: Understanding weights Exercise 7: Hidden layers and parameters Exercise 8: Your first neural network Exercise 9: Stacking linear layers Exercise 10: Counting the number of parameters

To train a neural network in PyTorch, you will first need to understand additional components, such as activation and loss functions. You will then realize that training a network requires minimizing that loss function, which is done by calculating gradients. You will learn how to use these gradients to update your model's parameters.

Exercise 1: Discovering activation functions Exercise 2: Activate your understanding!Exercise 3: The sigmoid and softmax functions Exercise 4: Running a forward pass Exercise 5: Building a binary classifier in PyTorch Exercise 6: From regression to multi-class classification Exercise 7: Using loss functions to assess model predictions Exercise 8: Creating one-hot encoded labels Exercise 9: Calculating cross entropy loss Exercise 10: Using derivatives to update model parameters Exercise 11: Accessing the model parameters Exercise 12: Updating the weights manually Exercise 13: Using the PyTorch optimizer

Now that you've learned the key components of a neural network, you'll train one using a training loop. You'll explore potential issues like vanishing gradients and learn strategies to address them, such as alternative activation functions and tuning learning rate and momentum.

Exercise 1: A deeper dive into loading data Exercise 2: Using TensorDataset Exercise 3: Using DataLoader Exercise 4: Writing our first training loop Exercise 5: Using the MSELoss Exercise 6: Writing a training loop Exercise 7: ReLU activation functions Exercise 8: Implementing ReLU Exercise 9: Implementing leaky ReLU Exercise 10: Understanding activation functions Exercise 11: Learning rate and momentum Exercise 12: Experimenting with learning rate Exercise 13: Experimenting with momentum

Training a deep learning model is an art, and to make sure our model is trained correctly, we need to keep track of certain metrics during training, such as the loss or the accuracy. We will learn how to calculate such metrics and how to reduce overfitting.

Exercise 1: Layer initialization and transfer learning Exercise 2: Fine-tuning process Exercise 3: Freeze layers of a model Exercise 4: Layer initialization Exercise 5: Evaluating model performance Exercise 6: Writing the evaluation loop Exercise 7: Calculating accuracy using torchmetrics Exercise 8: Fighting overfitting Exercise 9: Experimenting with dropout Exercise 10: Understanding overfitting Exercise 11: Improving model performance Exercise 12: Implementing random search Exercise 13: Wrap-up video