Setup up semantic masks

A common way to perform panoptic segmentation is to combine together the outputs of semantic and instance segmentation. Consider the following image of a New York street.

street image

Your task is to segment it panoptically, such that each cab is identified as a separate object, while distinguishing between the asphalt and building backgrounds.

To achieve it, you will start by producing a semantic mask with a pre-trained U-Net, available to you as UNet(). Hopefully, it should distinguish between the two background types (but not between particular cabs)!

Deze oefening maakt deel uit van de cursus

Deep Learning for Images with PyTorch

Cursus bekijken

Oefeninstructies

Instantiate the U-Net model as model.
Generate semantic_masks by passing the input image tensor to the model.
Create single semantic mask by choosing the highest-probability class for each pixel.

Praktische interactieve oefening

Probeer deze oefening eens door deze voorbeeldcode in te vullen.

# Instantiate the model
model = ____

# Produce semantic masks for the input image
with torch.no_grad():
    semantic_masks = ____

# Choose highest-probability class for each pixel
semantic_mask = ____(____, ____)

# Display the mask
plt.imshow(semantic_mask.squeeze(0))
plt.axis("off")
plt.show()

Code bewerken en uitvoeren