Motivation

Instructor note

  • 10 min teaching/discussion

  • 0 min exercises

Research comic

A scary anecdote

  • A group of researchers obtain great results and submit their work to a high-profile journal.

  • Reviewers ask for new figures and additional analysis.

  • The researchers start working on revisions and generate modified figures, but find inconsistencies with old figures.

  • The researchers can’t find some of the data they used to generate the original results, and can’t figure out which parameters they used when running their analyses.

  • The manuscript is still languishing in the drawer …


Why talking about reproducible research?

A 2016 survey in Nature revealed that irreproducible experiments are a problem across all domains of science:

reproduciblity Nature

This study is now few years old but the highlighted problem did not get smaller.


Levels of reproducibility

A published article is like the top of a pyramid. It rests on multiple levels that each contributes to its reproducibility.

Reproducibility pyramid

[citation needed]


6 helpful steps for reproducible research: file organization, naming, documentation, version control, stabilizing computing environment, publishing research outputs - figure by Heidi Seibold, CC-BY 4.0

[Heidi Seibold, CC-BY 4.0, https://twitter.com/HeidiBaya/status/1579385587865649153]


Discuss in collaborative document or with your team members

- What are your experiences re-running or adjusting a script or a figure you
  created few months ago?
  - ...
  - ...
  - (share your experience)

- Have you continued working from a previous student's
  script/code/plot/notebook? What were the biggest challenges?
  - ...
  - ...
  - (share your experience, but constructively)
Then a mirace occurs

[citation needed]