Start studying
Module 1 · Machine Learning Module contents
Module 1  ·  Card 24 / 55intermediate

Data Leakage

Data leakage is when information the model should not have sneaks into training.

A classic case is scaling or imputing using statistics computed over the whole dataset.

Another is including a feature that is only known after the outcome happened.

It produces amazing validation scores that collapse in production.

The fix is to fit every transformation on the training split only.

Learn in depth ↓
Module 01 · Machine Learning · 55 concepts
In depth  /  15 lines

Data Leakage, in more detail

Leakage is when the model sees something in training that it will not have later.

It makes your test scores look great and your real performance terrible.

The rule is simple: only use what would truly be known at prediction time.

The most common slip happens before you even start training.

You scale the numbers or fill in gaps using the whole dataset.

That quietly mixes information from your test rows into training.

Always split first, then fit every transformation on the training part only.

A worse kind is a column that only exists after the outcome happened.

A field like date the account closed will predict churn perfectly.

It also teaches the model nothing you can ever use.

Time is another trap whenever order matters.

Split at random and you end up training on the future and testing on the past.

For anything with dates, split by date instead.

Groups leak too, when the same person shows up on both sides of the split.

The blunt test: if a score looks too good, look for leakage before you celebrate.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
You scale your features using the mean of the entire dataset before splitting. What have you done?
Machine Learning

55 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close