Audioreiniging (denoising)

In deze oefening ga je data gebruiken uit de WHAM-gegevensset, waarin spraak wordt gemixt met achtergrondgeluid, om nieuwe spraak te genereren in een andere stem én zonder het achtergrondgeluid!

Spectrogram of noisy speech

De array example_speech en de vector speaker_embedding van de nieuwe stem zijn al geladen. De preprocessor (processor) en vocoder (vocoder) zijn ook beschikbaar, samen met de module SpeechT5ForSpeechToSpeech. Er is een functie make_spectrogram() meegeleverd om te helpen bij het plotten.

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

Cursus bekijken

Oefeninstructies

Laad het voorgetrainde model SpeechT5ForSpeechToSpeech met het microsoft/speecht5_vc-checkpoint.
Verwerk example_speech voor met een samplingfrequentie van 16000.
Genereer de opgeschoonde spraak met .generate_speech().

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

# Load the SpeechT5ForSpeechToSpeech pretrained model
model = ____

# Preprocess the example speech
inputs = ____(audio=____, sampling_rate=____, return_tensors="pt")

# Generate the denoised speech
speech = ____

make_spectrogram(speech)
sf.write("speech.wav", speech.numpy(), samplerate=16000)

Code bewerken en uitvoeren

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

SkillTag.level.intermediateSkillTag.label

4.8+

Begin de cursus gratis

Navigate the Hugging Face model hub, transform raw text, audio, and visual data into AI-friendly formats. Learn how to find the latest most popular models for tasks such as text generation and harness the power of pre-built pipelines.

Exercise 1: Hugging Face model navigation Exercise 2: How many models!?Exercise 3: Finding the most popular text-to-image model Exercise 4: Preprocessing different modalities Exercise 5: Text tokenizing Exercise 6: Image preprocessing Exercise 7: Audio preprocessing Exercise 8: Pipeline tasks and evaluations Exercise 9: Pipeline caption generation Exercise 10: Passing keyword arguments Exercise 11: Model evaluation on a custom dataset

Learn to master individual modalities with state-of-the-art models. Dive into computer vision for image classification and segmentation, explore speech recognition and text-to-speech synthesis, and learn effective fine-tuning techniques. Build practical skills with pre-trained models from Hugging Face's transformers library.

Exercise 1: Computer vision Exercise 2: Beeldclassificatie Exercise 3: Objectdetectie Exercise 4: Achtergrond van afbeelding verwijderen Exercise 5: Computer vision-modellen fine-tunen Exercise 6: CV fine-tuning: gegevensset voorbereiden Exercise 7: CV-finetuning: modelklassen Exercise 8: CV-fine-tuning: trainerconfiguratie Exercise 9: Spraakherkenning en audiogeneratie Exercise 10: Automatische spraakherkenning Exercise 11: Spraakembeddings maken Exercise 12: Audioreiniging (denoising)

Huidige oefening

Exercise 13: Fijn-tunen van text-to-speech-modellen Exercise 14: Een text-to-speechmodel fine-tunen Exercise 15: Nieuwe spraak genereren

Learn to fuse visual, textual, and audio information for richer AI applications. Master techniques like CLIP for zero-shot classification, build sentiment analyzers that see and read, and create emotion detectors that combine facial expressions with voice. Take your AI models beyond single-modality thinking.

Exercise 1: Zero-shot image classification Exercise 2: Zero-shot learning with CLIP Exercise 3: Automated caption quality assessment Exercise 4: Multi-modal sentiment analysis Exercise 5: Prompting Vision Language Models (VLMs)Exercise 6: Multi-modal sentiment classification with Qwen Exercise 7: Zero-shot video classification Exercise 8: Video audio splitting Exercise 9: Video sentiment analysis with CLIP CLAP

Transform ideas into reality! Master cutting-edge AI techniques to generate and manipulate visual content using text prompts. Create stunning images, edit photos intelligently, and build powerful question-answering systems for images and documents. Turn your creative vision into digital reality with multi-modal AI.

Exercise 1: Visual question-answering (VQA)Exercise 2: VQA with Vision Language Transformers (ViLTs)Exercise 3: Document VQA with LayoutLM Exercise 4: Image editing with diffusion models Exercise 5: Custom image editing Exercise 6: Image inpainting Exercise 7: Video generation Exercise 8: Build a video!Exercise 9: Assessing video generation performance Exercise 10: Congratulations!