Bir transformer modeli oluşturma

PyBooks'ta üzerinde çalıştığın öneri motorunun, kullanıcı yorumlarının duyarlılığını daha iyi anlamaya ihtiyacı var. Güncel mimarilerden olan transformer'ların bu hedefe ulaşmaya yardımcı olacağını düşünüyorsun. Projeyi başlatmak için yorumlardaki duyarlılıkları kodlayabilecek bir transformer modeli kurmaya karar verdin.

Senin için şu paketler içe aktarıldı: torch, nn, optim.

Girdi verisi şu tür cümleleri içeriyor: "I love this product", "This is terrible", "Could be better" … ve bunlara karşılık gelen ikili duyarlılık etiketleri: 1, 0, 0, ...

Girdi verisi bölünüp gömme vektörlerine dönüştürülerek şu değişkenlerde tutuluyor: train_sentences, train_labels ,test_sentences,test_labels,token_embeddings

Bu egzersiz

PyTorch ile Metin için Deep Learning

kursunun bir parçasıdır

Kursu Görüntüle

Egzersiz talimatları

Transformer kod çözücüyü başlat.
Duyarlılık sınıfı sayısına göre tam bağlı katmanı tanımla.
forward metodunda girdiyi önce transformer kod çözücüden, ardından doğrusal katmandan geçir.

Uygulamalı interaktif egzersiz

Bu örnek kodu tamamlayarak bu egzersizi bitirin.

class TransformerEncoder(nn.Module):
    def __init__(self, embed_size, heads, num_layers, dropout):
        super(TransformerEncoder, self).__init__()
        # Initialize the encoder 
        self.encoder = nn.____(
            nn.____(d_model=embed_size, nhead=heads),
            num_layers=num_layers)
        # Define the fully connected layer
        self.fc = nn.Linear(embed_size, ____)

    def forward(self, x):
        # Pass the input through the transformer encoder 
        x = self.____(x)
        x = x.mean(dim=1) 
        return self.fc(x)

model = TransformerEncoder(embed_size=512, heads=8, num_layers=3, dropout=0.5)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

Kodu Düzenle ve Çalıştır

Bu egzersiz

PyTorch ile Metin için Deep Learning

kursunun bir parçasıdır

AvançadoNível de habilidade

4.8+

Kursa Ücretsiz Başlayın

This chapter introduces you to deep learning for text and its applications. Learn how to use PyTorch for text processing and get hands-on experience with techniques such as tokenization, stemming, stopword removal, and more. Understand the importance of encoding text data and implement encoding techniques using PyTorch. Finally, consolidate your knowledge by building a text processing pipeline combining these techniques.

Exercise 1: Introduction to preprocessing for text Exercise 2: Word frequency analysis Exercise 3: Preprocessing text Exercise 4: Encoding text data Exercise 5: One-hot encoded book titles Exercise 6: Bag-of-words for book titles Exercise 7: Applying TF-IDF to book descriptions Exercise 8: Introduction to building a text processing pipeline Exercise 9: Shakespearean language preprocessing pipeline Exercise 10: Shakespearean language encoder

Explore text classification and its role in Natural Language Processing (NLP). Apply your skills to implement word embeddings and develop both Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) for text classification using PyTorch, and understand how to evaluate your models using suitable metrics.

Exercise 1: Overview of Text Classification Exercise 2: Embedding in PyTorch Exercise 3: Categorizing text classification tasks Exercise 4: Convolutional neural networks for text classification Exercise 5: Build a CNN model for text Exercise 6: Train a CNN model for text Exercise 7: Testing the Sentiment Analysis CNN Model Exercise 8: Recurrent neural networks for text classification Exercise 9: Building an RNN model for text Exercise 10: Building an LSTM model for text Exercise 11: Building a GRU model for text Exercise 12: Evaluation metrics for text classification Exercise 13: Evaluating RNN classification models Exercise 14: Evaluating the model's performance Exercise 15: Comparing models

Venture into the exciting world of text generation and its applications in NLP. Understand how to leverage Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and pre-trained models for text generation tasks using PyTorch. Alongside, you'll learn to evaluate the performance of your models using relevant metrics.

Exercise 1: Introduction to text generation Exercise 2: Creating a RNN model for text generation Exercise 3: Text generation using RNN - Training and Generation Exercise 4: Generative adversarial networks for text generation Exercise 5: Building a generator and discriminator Exercise 6: Training a GAN model Exercise 7: Pre-trained models for text generation Exercise 8: Text completion with pre-trained GPT-2 models Exercise 9: Language translation with pretrained PyTorch model Exercise 10: Evaluation metrics for text generation Exercise 11: Evaluating pretrained text generation model Exercise 12: Understanding text generation metrics

Understand the concept of transfer learning and its application in text classification. Explore Transformers, their architecture, and how to use them for text classification and generation tasks. You will also delve into attention mechanisms and their role in text processing. Finally, understand the potential impacts of adversarial attacks on text classification models and learn how to protect your models.

Exercise 1: Metin sınıflandırma için transfer learning Exercise 2: BERT ile transfer öğrenimi Exercise 3: BERT modelini değerlendirme Exercise 4: Metin işleme için Transformer'lar Exercise 5: Bir transformer modeli oluşturma

Geçerli Egzersiz

Exercise 6: Transformer modelini eğitme ve test etme Exercise 7: Metin işlemede dikkat mekanizmaları Exercise 8: Dikkat mekanizmalı bir RNN modeli oluşturma Exercise 9: Dikkat mekanizmalı RNN modelini eğitme ve test etme Exercise 10: Metin sınıflandırma modellerine karşı adversaryal saldırılar Exercise 11: Adversarial saldırı sınıflandırma Exercise 12: PyBooks'ta AI'ı Koruma Exercise 13: Kapanış