TracksStudyAbout
Sign in
Start studying
Module 1 · Machine Learning Module contents
Module 1  ·  Card 51 / 55beginner

MSE

MSE is the mean squared error for regression.

You square each error before averaging them.

Squaring makes large mistakes count much more heavily.

That makes it sensitive to outliers.

Its units are the square of the target, which is awkward to interpret.

Back Next
Learn in depth ↓
Module 01 · Machine Learning · 55 concepts
In depth  /  15 lines

MSE, in more detail

MSE squares each error before averaging them.

Squaring throws away the sign, the same as taking the absolute value.

But it also makes large errors count far more than small ones.

An error of ten contributes a hundred times as much as an error of one.

So the model is pushed hard to avoid any single disaster.

That is useful when one big miss really is much worse than several small ones.

It is also mathematically convenient, which is why so many optimisers use it.

The curve is smooth everywhere, which keeps gradient descent well behaved.

The cost is heavy sensitivity to outliers.

One mislabelled row can pull the whole model toward it.

Its units are the square of the target, which nobody can interpret.

Squared pounds and squared degrees mean nothing to a reader.

That is why it is used for training and rarely for reporting.

Check your data for outliers before choosing it as the loss.

If a few extreme rows are mistakes rather than reality, clean them first.

Want the full technical article? Wikipedia ↗
Deep dive

What is underneath MSE

The formula

MSE=1n∑i=1n(yi−y^i)2\mathrm{MSE} = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2

Squaring does two jobs at once: it removes the sign, and it makes large errors count for much more than small ones.

An error of 10 contributes 100. An error of 1 contributes 1. One bad prediction can outweigh a hundred good ones.

The units are the target's units squared, which is why MSE is hard to interpret directly and why RMSE exists.

Why it is the default loss

∂∂y^(y−y^)2=−2(y−y^)\frac{\partial}{\partial \hat{y}} (y - \hat{y})^2 = -2(y - \hat{y})

The gradient is proportional to the error itself, which is exactly the behaviour you want from an optimiser.

Far from the answer it pushes hard. Close to the answer it eases off, so the parameters settle rather than oscillating.

It is also smooth everywhere, including at zero, unlike the absolute error.

What it is estimating

arg⁡min⁡c  E[(Y−c)2]=E[Y]\arg\min_{c} \; \mathbb{E}\left[ (Y - c)^2 \right] = \mathbb{E}[Y]

Minimising squared error gives you the mean of the conditional distribution.

There is a deeper reason it is the default: minimising squared error is the maximum-likelihood estimate under the assumption that the noise is Gaussian with constant variance.

So choosing MSE is not a neutral act. It asserts something about the shape of the noise, and that assertion is usually left unstated.

The outlier problem

  1. One label is mistyped: 1,000,000 instead of 100.
  2. The error is roughly 999,900.
  3. Squared, it contributes about 101210^{12} to the sum.
  4. That single point now outweighs the entire rest of the dataset.
  5. The fit bends towards it, and every other prediction gets worse.

This is not a hypothetical. It is the most common way a regression model is quietly ruined, and the fix is to clean the labels or to switch to a robust loss such as Huber.

Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
Why is MSE awkward to report to a non-technical reader?
Concepts these five lines lean on
Gradient DescentRecallOptimizationRegressionLoss FunctionOutliers · Data ScienceMean · Data Science
Machine Learning

55 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close