Motivation

Instructor note

10 min teaching/discussion
0 min exercises

A scary anecdote

A group of researchers obtain great results and submit their work to a high-profile journal.
Reviewers ask for new figures and additional analysis.
The researchers start working on revisions and generate modified figures, but find inconsistencies with old figures.
The researchers can’t find some of the data they used to generate the original results, and can’t figure out which parameters they used when running their analyses.
The manuscript is still languishing in the drawer …

Why talking about reproducible research?

A 2016 survey in Nature revealed that irreproducible experiments are a problem across all domains of science:

This study is now few years old but the highlighted problem did not get smaller.

Levels of reproducibility

A published article is like the top of a pyramid. It rests on multiple levels that each contributes to its reproducibility.

[citation needed]

6 helpful steps for reproducible research: file organization, naming, documentation, version control, stabilizing computing environment, publishing research outputs - figure by Heidi Seibold, CC-BY 4.0

[Heidi Seibold, CC-BY 4.0, https://twitter.com/HeidiBaya/status/1579385587865649153]

Discuss in collaborative document or with your team members

- What are your experiences re-running or adjusting a script or a figure you
  created few months ago?
  - ...
  - ...
  - (share your experience)

- Have you continued working from a previous student's
  script/code/plot/notebook? What were the biggest challenges?
  - ...
  - ...
  - (share your experience, but constructively)

[citation needed]