Image inpainting

Let's put a spin on multi-modal image generation by combining it with image inpainting. You will modify the self-portrait of Van Gogh to have a black beard using the StableDiffusionControlNetInpaintPipeline and an image mask, which has been created for you (mask_image):

Famous Van Gogh painting and mask

Note: Inference on diffusion models can take a long time, so we've pre-loaded the generated image for you. Running different prompts will not generated new images.

The original version of the image has been loaded as init_image, along with a control image (control_image) created with the make_inpaint_condition() function from the video.

Run the pipeline with a prompt designed to generate a black beard, specifying num_inference_steps=40, and passing the init_image, mask_image, and control_image.

Exercise

Image inpainting

Instructions

.css-6su6fj{-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;}Exercise

Instructions

Exercise