LoslegenKostenlos loslegen

Image inpainting

Let's put a spin on multi-modal image generation by combining it with image inpainting. You will modify the self-portrait of Van Gogh to have a black beard using the StableDiffusionControlNetInpaintPipeline and an image mask, which has been created for you (mask_image):

Famous Van Gogh painting and mask

Note: Inference on diffusion models can take a long time, so we've pre-loaded the generated image for you. Running different prompts will not generated new images.

The original version of the image has been loaded as init_image, along with a control image (control_image) created with the make_inpaint_condition() function from the video.

Diese Übung ist Teil des Kurses

Multi-Modal Models with Hugging Face

Kurs anzeigen

Anleitung zur Übung

  • Run the pipeline with a prompt designed to generate a black beard, specifying num_inference_steps=40, and passing the init_image, mask_image, and control_image.

Interaktive Übung

Vervollständige den Beispielcode, um diese Übung erfolgreich abzuschließen.

# Run the pipeline requesting a black beard
output = pipe(
    ____,
    num_inference_steps=____,
    eta=1.0,
    image=____,
    mask_image=____,
    control_image=____
)

plt.imshow(output.images[0])
plt.show()
Code bearbeiten und ausführen