Model compression shrinks a model while preserving as much capability as possible.
Pruning removes weights or whole structures that contribute little.
Quantization lowers precision and distillation trains a smaller replacement.
These techniques stack and are frequently combined.
The aim is cheaper, faster inference within a quality budget you set.
Model compression makes a model smaller and faster while keeping as much capability as possible.
It matters because the cost of a model is paid at every single request.
A model that is half the size can be twice as cheap forever.
There are three main techniques and they attack different things.
Quantization lowers the precision of each number.
Pruning removes weights or structures that contribute little.
Distillation trains a smaller model to imitate the original.
They stack, so a pruned, quantised, distilled model is entirely normal.
Architecture choice is the fourth lever and often the largest.
A model designed to be small beats a large one squeezed down.
The aim is not the smallest model but the cheapest one that clears your quality bar.
So decide the bar first and compress until you approach it.
Measure quality on your own task, since benchmark scores hide task-specific damage.
Reasoning and long-context ability degrade before simple tasks do.
Measure real latency on the real hardware, because theoretical savings often fail to appear.
One question on this card. Every option explains itself once you pick, right or wrong.
23 concepts in this module, then a graded test with every answer explained.