A loss function measures how wrong a single prediction is.
If the model says 80 and the true answer is 100, there is an error of 20.
The loss function turns that error into one number the computer can work with.
Training tries to find parameters that make the total loss as small as possible.
Different problems use different losses, such as MSE for regression and cross-entropy for classification.
The loss function is how the model knows it is wrong.
Without it there is no direction to improve in.
It takes a prediction and the truth and returns one number.
Bigger means worse, and zero means perfect.
Training is simply the search for the smallest number it can reach.
Which loss you choose changes what the model cares about.
Squared error cares enormously about large mistakes.
Absolute error treats a miss of ten as ten times a miss of one.
So squared error chases outliers while absolute error mostly ignores them.
For classification the usual choice is cross-entropy.
It rewards being confident and right, and punishes being confident and wrong.
Being unsure and wrong is punished far more gently.
The loss is what you optimise, but not always what you actually care about.
You might optimise cross-entropy and then be judged on F1.
When the two disagree, the model follows the loss and not your intentions.
One question on this card. Every option explains itself once you pick, right or wrong.
55 concepts in this module, then a graded test with every answer explained.