Setup up semantic masks
A common way to perform panoptic segmentation is to combine together the outputs of semantic and instance segmentation. Consider the following image of a New York street.
Your task is to segment it panoptically, such that each cab is identified as a separate object, while distinguishing between the asphalt and building backgrounds.
To achieve it, you will start by producing a semantic mask with a pre-trained U-Net, available to you as UNet(). Hopefully, it should distinguish between the two background types (but not between particular cabs)!
This exercise is part of the course
Deep Learning for Images with PyTorch
Exercise instructions
- Instantiate the U-Net model as
model. - Generate
semantic_masksby passing the input image tensor to the model. - Create single semantic mask by choosing the highest-probability class for each pixel.
Hands-on interactive exercise
Have a go at this exercise by completing this sample code.
# Instantiate the model
model = ____
# Produce semantic masks for the input image
with torch.no_grad():
semantic_masks = ____
# Choose highest-probability class for each pixel
semantic_mask = ____(____, ____)
# Display the mask
plt.imshow(semantic_mask.squeeze(0))
plt.axis("off")
plt.show()