Start studying
Module 12 · Time Series Module contents
Module 12  ·  Card 12 / 13advanced

Time Series Cross-Validation

Time series cross-validation tests a forecasting method by always fitting on the past and testing on the future.

Random splits leak future information into fitting, which makes errors look far smaller than they will be.

For example, a model is fitted on January to June and tested on July, then fitted up to July and tested on August.

This rolling approach, also called backtesting, shows how the method would have performed in real use.

Each test window should match the forecasting horizon, so a weekly forecast is tested on the following week.

Learn in depth ↓
Module 12 · Time Series · 13 concepts
In depth  /  15 lines

Time Series Cross-Validation, in more detail

Time series cross-validation tests a forecasting method the way it will be used: on unseen data from the future.

It exists because ordinary random splits break the order of time.

A random split lets the model learn from next month to predict last month, which no real forecast can do.

The expanding window approach fits on all data up to a point and tests on the period after it.

The point then moves forward and the process repeats, giving many test results.

The sliding window approach uses a fixed length of recent history, which suits series whose behaviour changes.

Each test window should match the forecasting horizon in use.

A gap between the fitting and test periods can mimic the delay before data becomes available.

Averaging errors across all the test windows gives a fair estimate of future accuracy.

Looking at errors window by window shows whether the method is getting worse over time.

Any step that uses future information, such as scaling by the whole series' mean, leaks and must happen inside each window.

Inputs built from future values, such as a centred moving average, leak in the same way.

Backtests are only as good as the history they cover, so a calm period hides how a method fails in a shock.

In practice, comparing every method against a simple baseline in the same backtest keeps results honest.

The usual mistake is reporting accuracy from a random split, which can make a forecast look far better than it will be.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A forecasting model is tested by splitting 3 years of daily data into random fitting days and test days. What is wrong with this?
Time Series

13 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close