Dataset evaluation checks the quality of the data itself.
You look for label noise, duplicates, imbalance and missing values.
You also check that it represents the population you will actually serve.
Duplicates between train and test silently inflate your scores.
Bad data caps performance no matter how good the model is.
Before evaluating the model, evaluate the data, since the data caps everything.
No architecture recovers from labels that are wrong.
Label noise is the first thing to look for and the most commonly ignored.
Have two people label the same hundred rows and see how often they agree.
If humans agree only 80 per cent of the time, no model will exceed that by much.
Duplicates are the second thing, and the dangerous ones straddle the train and test split.
A row appearing in both means the model is scored on something it memorised.
That single mistake produces impossible-looking results more often than any other.
Check near-duplicates too, since a reworded sentence is effectively the same row.
Look at the class balance and at how missing values are distributed.
Ask whether the data represents the population you will actually serve.
A model trained on one region's customers will not transfer quietly to another.
Look at the collection date, since old data describes a world that has moved on.
Read a hundred rows by hand, because problems are visible and rarely caught by script.
Time spent here beats time spent tuning by a wide margin.
One question on this card. Every option explains itself once you pick, right or wrong.
18 concepts in this module, then a graded test with every answer explained.