Exercise

Assessing video generation performance

You can assess the performance of your video generation pipelines using a multi-modal CLIP model, which tests the similarity between each video frame image and the prompt. You will use this to assess just how well your generated video from the previous exercise matches the prompt.

The load_video() function has been imported from diffusers.utils for you. The clip_score module has also been imported from torchmetrics.

Instructions

100 XP

Set up a CLIP scoring function called clip_score_fn() from the clip_score() metric.
Calculate the CLIP score between each frame tensor in frame_tensors and prompt.

.css-6su6fj{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;}Exercise

Instructions

Exercise