Overlay instance masks

Good job producing the semantic mask in the previous exercise! Now, you can overwrite it with instance masks in the locations where the objects have been identified by the instance segmentation model.

You will use the pre-trained MaskRCNN available in your workspace to produce instance segmentation masks. Then, you will loop over these masks and for each of them, you will overlay the parts where an object is detected with high certainty on top of the semantic mask.

torch is already imported for you.

This exercise is part of the course

Deep Learning for Images with PyTorch

View Course

Exercise instructions

Initialize panoptic_mask by cloning the semantic_mask.
Define the for-loop to iterate over the instance masks, calling the iterator variable mask.
For each instance mask, in location where it is larger than 0.5, overwrite the panoptic mask with the current instance_id.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Instantiate model and produce instance masks
model = MaskRCNN()
with torch.no_grad():
    instance_masks = model(image_tensor)[0]["masks"]

# Initialize panoptic mask as semantic_mask
panoptic_mask = ____

# Iterate over instance masks
instance_id = 3
for ____ in ____:
    # Set panoptic mask to instance_id where mask > 0.5
    panoptic_mask[____] = ____
    instance_id += 1
    
# Display panoptic mask
plt.imshow(panoptic_mask.squeeze(0))
plt.axis("off")
plt.show()

Edit and Run Code