Reproducible research software development using Python

Big-picture goal

This is a hands-on course on research software engineering. In this workshop we assume that most workshop participants use Python in their work or a leading a group which uses Python. Therefore, some of the examples will use Python as the example language.

We will work with an example project and go through all important steps of a typical software project. Once we have seen the building blocks, we will try to apply them to own projects.

Prerequisites

Preparation

1. Get a GitHub account following these instructions.

2. You will need a text editor. If you don’t have a favorite one, we recommend VS Code.

3. If you prefer to work in the terminal and not in VS Code, set up these two (skip this if you use VS Code):

   • Git in the terminal

   • SSH or HTTPS connection to GitHub from terminal

4. One of these two software environments (if you are not sure which one to choose or have no preference, choose Conda):

   • Conda environment

   • Virtual environment (Snakemake is not available in this environment)

5. Optional and only on Linux: Apptainer following these instructions.

Schedule

Day 1

• 13:00-13:30 - Welcome and introduction

  • Practical information (tools, communication, breaks, etc.)

  • Motivation (reproducibility, robustness, distribution, improvement, trust, etc.)

  • Example project: Simulating the motion of planets

• 13:30-14:45 - Introduction to version control with Git and GitHub (1/2)

  • Motivation (15 min)

  • Forking, cloning, and browsing (30 min)

  • Creating branches and commits (30 min)

• 15:00-16:30 - Introduction to version control with Git and GitHub (2/2)

  • Merging changes and contributing to the project (40 min)

  • Conflict resolution (30 min)

  • Practical advice: How much Git is necessary? (20 min)

  • Optional: How to turn your project to a Git repo and share it (30 min)

• 16:45-18:00 - Code documentation

Day 2

• 09:00-10:30 - Collaborative version control and code review (1/2)

  • Concepts around collaboration (10 min)

  • Collaborating within the same repository (40 min)

  • Practicing code review (40 min)

• 10:45-12:15 - Collaborative version control and code review (2/2)

  • How to contribute changes to repositories that belong to others (50 min)

  • Conflict resolution, rebasing, and organizational strategies (40 min)

• 16:45-18:00 - Debriefing and Q&A

  • Participants work on their projects

  • Together we study actual codes that participants wrote or work on

  • Constructively we discuss possible improvements

  • Give individual feedback on code projects

Day 3

• 09:00-10:30 - Automated testing

• 10:45-12:15 - How to make the project more reusable

• 13:00-14:45 - Code quality and good practices

  • Concepts in refactoring and modular code design (15 min)

  • Demo: From a script towards a workflow (90 min)

• 15:00-16:30 - How to release and publish your code

  • Choosing a software license (30 min)

  • How to publish your code (15 min)

  • Creating a Python package and deploying it to PyPI (45 min)

• 16:45-18:00 - Debriefing and Q&A

  • Participants work on their projects

  • Together we study actual codes that participants wrote or work on

  • Constructively we discuss possible improvements

  • Give individual feedback on code projects