Image Segmentation

Segmentation labels every pixel rather than drawing a box around a region.

A box says roughly where a tumour is, and a mask says exactly which pixels it occupies.

That precision is the whole reason the technique exists.

Semantic segmentation gives each pixel a class, such as road, car or sky.

It does not separate one car from another, since both are simply car.

Instance segmentation separates individual objects of the same class.

Panoptic segmentation combines both, labelling countable things and uncountable stuff together.

The output is a mask the same size as the input, which forces a particular architecture.

The network must downsample to gain context and then upsample back to full resolution.

Skip connections carry fine detail across that gap, which is what a U-Net does.

Labelling training data is expensive, since a person must outline every object.

That expense is why medical segmentation datasets are small and augmentation matters so much.

Dice score and IoU are the usual metrics, computed over pixels rather than boxes.

Medical imaging, satellite analysis and self-driving perception all depend on it.

Segment Anything and similar foundation models now produce masks with little or no task-specific training.