Crea un video!

È il tuo turno: prova a creare un video partendo solo da un prompt testuale! Userai una pipeline CogVideoXPipeline e il seguente prompt per guidarne la generazione:

A robot doing the robot dance. The dance floor has colorful squares and a glitterball.

Nota: l'inferenza con modelli di generazione video può richiedere molto tempo, quindi abbiamo già precaricato per te il video generato. Eseguire prompt diversi non genererà nuovi video.

La classe CogVideoXPipeline è già stata importata per te.

Questo esercizio fa parte del corso

Modelli multi-modali con Hugging Face

Visualizza il corso

Istruzioni dell'esercizio

Crea una CogVideoXPipeline dal checkpoint THUDM/CogVideoX-2b.
Esegui la pipeline con il prompt fornito, impostando il numero di passi di inferenza a 20, il numero di frame da generare a 20 e il guidance scale a 6.

Esercizio pratico interattivo

Prova a risolvere questo esercizio completando il codice di esempio.

prompt = "A robot doing the robot dance. The dance floor has colorful squares and a glitterball."

# Create a CogVideoXPipeline
pipe = ____(
    "____",
    torch_dtype=torch.float16
)

# Run the pipeline with the provided prompt
video = pipe(
    prompt=____,
    num_inference_steps=____,
    num_frames=____,
    guidance_scale=____
)
video = video.frames[0]

video_path = export_to_video(video, "output.mp4", fps=8)
video = VideoFileClip(video_path)
video.write_gif("video_ex.gif")

Modifica ed esegui il codice

Questo esercizio fa parte del corso

Modelli multi-modali con Hugging Face

IntermediárioNível de habilidade

4.8+

Inizia il corso gratis

Navigate the Hugging Face model hub, transform raw text, audio, and visual data into AI-friendly formats. Learn how to find the latest most popular models for tasks such as text generation and harness the power of pre-built pipelines.

Exercise 1: Hugging Face model navigation Exercise 2: How many models!?Exercise 3: Finding the most popular text-to-image model Exercise 4: Preprocessing different modalities Exercise 5: Text tokenizing Exercise 6: Image preprocessing Exercise 7: Audio preprocessing Exercise 8: Pipeline tasks and evaluations Exercise 9: Pipeline caption generation Exercise 10: Passing keyword arguments Exercise 11: Model evaluation on a custom dataset

Learn to master individual modalities with state-of-the-art models. Dive into computer vision for image classification and segmentation, explore speech recognition and text-to-speech synthesis, and learn effective fine-tuning techniques. Build practical skills with pre-trained models from Hugging Face's transformers library.

Exercise 1: Computer vision Exercise 2: Image classification Exercise 3: Object detection Exercise 4: Image background removal Exercise 5: Fine-tuning computer vision models Exercise 6: CV fine-tuning: dataset prep Exercise 7: CV fine-tuning: model classes Exercise 8: CV fine-tuning: trainer configuration Exercise 9: Speech recognition and audio generation Exercise 10: Automatic speech recognition Exercise 11: Creating speech embeddings Exercise 12: Audio denoising Exercise 13: Fine-tuning text-to-speech models Exercise 14: Fine-tuning a text-to-speech model Exercise 15: Generating new speech

Learn to fuse visual, textual, and audio information for richer AI applications. Master techniques like CLIP for zero-shot classification, build sentiment analyzers that see and read, and create emotion detectors that combine facial expressions with voice. Take your AI models beyond single-modality thinking.

Exercise 1: Zero-shot image classification Exercise 2: Zero-shot learning with CLIP Exercise 3: Automated caption quality assessment Exercise 4: Multi-modal sentiment analysis Exercise 5: Prompting Vision Language Models (VLMs)Exercise 6: Multi-modal sentiment classification with Qwen Exercise 7: Zero-shot video classification Exercise 8: Video audio splitting Exercise 9: Video sentiment analysis with CLIP CLAP

Transform ideas into reality! Master cutting-edge AI techniques to generate and manipulate visual content using text prompts. Create stunning images, edit photos intelligently, and build powerful question-answering systems for images and documents. Turn your creative vision into digital reality with multi-modal AI.

Exercise 1: Visual question answering (VQA)Exercise 2: VQA con i Vision Language Transformers (ViLT)Exercise 3: VQA su documenti con LayoutLM Exercise 4: Modifica delle immagini con modelli di diffusione Exercise 5: Modifica personalizzata di immagini Exercise 6: Image inpainting Exercise 7: Generazione di video Exercise 8: Crea un video!

Esercizio in corso

Exercise 9: Valutare le prestazioni della generazione video Exercise 10: Congratulazioni!