Start studying
Module 14 · Data Science in Practice Module contents
Module 14  ·  Card 11 / 12intermediate

Reproducibility in Data Science

Reproducibility means someone else can rerun an analysis from the same data and code and get the same results.

It needs the raw data or a fixed snapshot of it, the code for every step, and a record of the software used.

For example, a notebook that depends on a file on one person's laptop and on cells run out of order cannot be reproduced.

Fixing random seeds, keeping code in a version control system such as Git, and scripting every step all help.

Reproducible work is easier to check, to update with new data and to trust.

Learn in depth ↓
Module 14 · Data Science in Practice · 12 concepts
In depth  /  15 lines

Reproducibility in Data Science, in more detail

Reproducibility turns an analysis from a one-off claim into something others can check and build on.

Results that cannot be rerun cannot be verified, updated or trusted for long.

The usual test is whether someone else can get the same numbers from the same inputs.

Raw data should be kept unchanged, with every cleaning step done in code.

Snapshots or dated copies of data stop a changing source from changing results.

Code should run from start to finish without manual steps, such as clicking or copying values.

Notebooks need restarting and running from the top before results are shared.

Random steps, such as sampling, need a fixed seed so they give the same result each time.

Recording package versions, for example in a requirements file, stops software updates from changing results.

Version control, such as Git, records every change to the code and who made it.

Containers can capture the whole software setup, though they add effort.

A clear folder layout and a short readme explain how to run the work.

Reproducibility is not the same as being correct, since the same mistake can be reproduced perfectly.

In practice, teams treat a report that cannot be regenerated as a draft, not a result.

The usual mistake is fixing data by hand in a spreadsheet, a step that leaves no trace and cannot be repeated.

Want the full technical article? Wikipedia ↗
Test yourself

One question on this card. Every option explains itself once you pick, right or wrong.

Question 1
A colleague reruns an analyst's notebook from the top and gets different numbers from the published report. What is the most likely cause?
Data Science in Practice

12 concepts in this module, then a graded test with every answer explained.

Study this module →
↑↓ move↵ openesc close