Nieuwe spraak genereren

Tijd om je beheersing van Hugging Face-audiomodellen compleet te maken! Je gaat een fijn-afgesteld model gebruiken om nieuwe spraak te genereren voor een gegeven stem. Je kiest een stem uit het VCTK Corpus als basis voor het nieuwe audiobestand.

De dataset en het SpeechT5ForTextToSpeech-model (model) zijn al geladen, en een make_spectogram()-functie is meegeleverd om te helpen bij het plotten.

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

Cursus bekijken

Oefeninstructies

Laad een voorbeeld van een speaker embedding van index 5 uit de test-dataset.
Genereer de spraak uit de verwerkte tekst door inputs, speaker_embedding en vocoder op te geven.

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

text = "Hi, welcome to your new voice."

# Load a speaker embedding from the dataset
speaker_embedding = torch.tensor(dataset[5]["____"]).unsqueeze(0)

inputs = processor(text=text, return_tensors="pt")
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")

# Generate speech
speech = model.generate_speech(____["input_ids"], ____, ____=____)

make_spectrogram(speech)

Code bewerken en uitvoeren

Deze oefening maakt deel uit van de cursus

Multi-modale modellen met Hugging Face

SkillTag.level.intermediateSkillTag.label

4.8+

Begin de cursus gratis

Navigate the Hugging Face model hub, transform raw text, audio, and visual data into AI-friendly formats. Learn how to find the latest most popular models for tasks such as text generation and harness the power of pre-built pipelines.

Exercise 1: Hugging Face model navigation Exercise 2: How many models!?Exercise 3: Finding the most popular text-to-image model Exercise 4: Preprocessing different modalities Exercise 5: Text tokenizing Exercise 6: Image preprocessing Exercise 7: Audio preprocessing Exercise 8: Pipeline tasks and evaluations Exercise 9: Pipeline caption generation Exercise 10: Passing keyword arguments Exercise 11: Model evaluation on a custom dataset

Learn to master individual modalities with state-of-the-art models. Dive into computer vision for image classification and segmentation, explore speech recognition and text-to-speech synthesis, and learn effective fine-tuning techniques. Build practical skills with pre-trained models from Hugging Face's transformers library.

Exercise 1: Computer vision Exercise 2: Beeldclassificatie Exercise 3: Objectdetectie Exercise 4: Achtergrond van afbeelding verwijderen Exercise 5: Computer vision-modellen fine-tunen Exercise 6: CV fine-tuning: gegevensset voorbereiden Exercise 7: CV-finetuning: modelklassen Exercise 8: CV-fine-tuning: trainerconfiguratie Exercise 9: Spraakherkenning en audiogeneratie Exercise 10: Automatische spraakherkenning Exercise 11: Spraakembeddings maken Exercise 12: Audioreiniging (denoising)Exercise 13: Fijn-tunen van text-to-speech-modellen Exercise 14: Een text-to-speechmodel fine-tunen Exercise 15: Nieuwe spraak genereren

Huidige oefening

Learn to fuse visual, textual, and audio information for richer AI applications. Master techniques like CLIP for zero-shot classification, build sentiment analyzers that see and read, and create emotion detectors that combine facial expressions with voice. Take your AI models beyond single-modality thinking.

Exercise 1: Zero-shot image classification Exercise 2: Zero-shot learning with CLIP Exercise 3: Automated caption quality assessment Exercise 4: Multi-modal sentiment analysis Exercise 5: Prompting Vision Language Models (VLMs)Exercise 6: Multi-modal sentiment classification with Qwen Exercise 7: Zero-shot video classification Exercise 8: Video audio splitting Exercise 9: Video sentiment analysis with CLIP CLAP

Transform ideas into reality! Master cutting-edge AI techniques to generate and manipulate visual content using text prompts. Create stunning images, edit photos intelligently, and build powerful question-answering systems for images and documents. Turn your creative vision into digital reality with multi-modal AI.

Exercise 1: Visual question-answering (VQA)Exercise 2: VQA with Vision Language Transformers (ViLTs)Exercise 3: Document VQA with LayoutLM Exercise 4: Image editing with diffusion models Exercise 5: Custom image editing Exercise 6: Image inpainting Exercise 7: Video generation Exercise 8: Build a video!Exercise 9: Assessing video generation performance Exercise 10: Congratulations!