Voorbewerking van audio

In deze oefening leer je hoe je de samplingfrequentie van audiodata aanpast en hoe je een automatische preprocessor gebruikt. Je werkt met het VCTK Corpus, dat ongeveer 44 uur aan spraakdata bevat, ingesproken door 110 Engelstalige sprekers met verschillende accenten.

De dataset is al geladen.

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

Cursus bekijken

Oefeninstructies

Resample de audio in de gegevensset naar een frequentie van 16.000 Hz met de methode .cast_column().
Laad de audioprocessor met het voorgetrainde model openai/whisper-small.
Verwerk de audiodata van het eerste datapunt voor, met dezelfde samplingfrequentie en padding=True.

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

# Resample the audio to a frequency of 16,000 Hz
dataset = dataset.____("____", ____(sampling_rate=____))

# Load the audio processor
processor = ____

# Preprocess the audio data of the 0th dataset element
audio_pp = ____(dataset[0]["audio"]["array"], sampling_rate=____, padding=True, return_tensors="pt")
make_spectrogram(audio_pp["input_features"][0])

Code bewerken en uitvoeren

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

SkillTag.level.intermediateSkillTag.label

4.8+

Begin de cursus gratis

Navigate the Hugging Face model hub, transform raw text, audio, and visual data into AI-friendly formats. Learn how to find the latest most popular models for tasks such as text generation and harness the power of pre-built pipelines.

Exercise 1: Navigeren door Hugging Face-modellen Exercise 2: Hoeveel modellen!?Exercise 3: Het populairste text-to-image-model vinden Exercise 4: Voorbewerking van verschillende modaliteiten Exercise 5: Tekst tokenizen Exercise 6: Afbeeldingen preprocessen Exercise 7: Voorbewerking van audio

Huidige oefening

Exercise 8: Pipelinetaken en evaluaties Exercise 9: Pipeline voor bijschriftgeneratie Exercise 10: Keyword-argumenten doorgeven Exercise 11: Model evalueren op een aangepaste gegevensset

Learn to master individual modalities with state-of-the-art models. Dive into computer vision for image classification and segmentation, explore speech recognition and text-to-speech synthesis, and learn effective fine-tuning techniques. Build practical skills with pre-trained models from Hugging Face's transformers library.

Exercise 1: Computer vision Exercise 2: Image classification Exercise 3: Object detection Exercise 4: Image background removal Exercise 5: Fine-tuning computer vision models Exercise 6: CV fine-tuning: dataset prep Exercise 7: CV fine-tuning: model classes Exercise 8: CV fine-tuning: trainer configuration Exercise 9: Speech recognition and audio generation Exercise 10: Automatic speech recognition Exercise 11: Creating speech embeddings Exercise 12: Audio denoising Exercise 13: Fine-tuning text-to-speech models Exercise 14: Fine-tuning a text-to-speech model Exercise 15: Generating new speech

Learn to fuse visual, textual, and audio information for richer AI applications. Master techniques like CLIP for zero-shot classification, build sentiment analyzers that see and read, and create emotion detectors that combine facial expressions with voice. Take your AI models beyond single-modality thinking.

Exercise 1: Zero-shot image classification Exercise 2: Zero-shot learning with CLIP Exercise 3: Automated caption quality assessment Exercise 4: Multi-modal sentiment analysis Exercise 5: Prompting Vision Language Models (VLMs)Exercise 6: Multi-modal sentiment classification with Qwen Exercise 7: Zero-shot video classification Exercise 8: Video audio splitting Exercise 9: Video sentiment analysis with CLIP CLAP

Transform ideas into reality! Master cutting-edge AI techniques to generate and manipulate visual content using text prompts. Create stunning images, edit photos intelligently, and build powerful question-answering systems for images and documents. Turn your creative vision into digital reality with multi-modal AI.

Exercise 1: Visual question-answering (VQA)Exercise 2: VQA with Vision Language Transformers (ViLTs)Exercise 3: Document VQA with LayoutLM Exercise 4: Image editing with diffusion models Exercise 5: Custom image editing Exercise 6: Image inpainting Exercise 7: Video generation Exercise 8: Build a video!Exercise 9: Assessing video generation performance Exercise 10: Congratulations!