Recording environments


  • Understand what containers are

  • Understand good and less good usecases for containers

  • Discuss container definitions files in the context of reusability and reproducibility

Instructor note

  • 10 min teaching/discussion

  • 20 min exercise

What is a container?

Imagine if you didn’t have to install things yourself, but instead you could get a computer with the exact software for a task pre-installed? Containers effectively do that, with various advantages and disadvantages. They are like an entire operating system with software installed, all in one file,

He said, then we will ship your machine. And that's how Docker was born.

From reddit.

Kitchen analogy

  • Our codes/scripts <-> cooking recipes

  • Container definition files <-> like a blueprint to build a kitchen with all utensils in which the recipe can be prepared.

  • Container images <-> example kitchens

  • Containers <-> identical factory-built mobile food truck kitchens

Just for fun: which operating systems do the following example kitchens represent?

Generated image of a kitchen

[Midjourney, CC-BY-NC 4.0]

From definition files to container images to containers

  • Containers can be built to bundle all the necessary ingredients (data, code, environment, operating system).

  • A container image is like a piece of paper with all the operating system on it. When you run it, a transparent sheet is placed on top to form a container. The container runs and writes only on that transparent sheet (and what other mounts have been layered on top). When you are done, transparency is thrown away. It can be repeated as often as you want, and base is always the same.

  • Definition files (e.g. Dockerfile or Singularity definition file) are text files that contain a series of instructions to build container images.

You may have use for containers in different ways

  • Installing a certain software is tricky, or not supported for your operating system? -> See if an image is available and run the software from a container instead!

  • You want to make sure your colleagues are using the same environment for running your code? -> Provide them an image of your container!

    • If this does not work, because they are using a different architecture than you do? -> Provide a definition file for them to build the image suitable to their computers. This does not create the exact environment as you have, but in most cases similar enough.

The container recipe

Here is an example of a Singularity definition file (reference):

Bootstrap: docker
From: ubuntu:20.04

    apt-get -y update
    apt-get -y install cowsay lolcat

    export LC_ALL=C
    export PATH=/usr/games:$PATH

    date | cowsay | lolcat

Popular container implementations:

They are to some extent interoperable:

  • podman is very close to Docker

  • Docker images can be converted to Singularity/Apptainer images

  • Singularity Python can convert Dockerfiles to Singularity definition files

Pros and cons of containers

Containers are popular for a reason - they solve a number of important problems:

  • Allow for seamlessly moving workflows across different platforms.

  • Can solve the “works on my machine” situation.

  • For software with many dependencies, in turn with its own dependencies, containers offer possibly the only way to preserve the computational experiment for future reproducibility.

  • A mechanism to “send the computer to the data” when data is too

  • Installing software into a file instead of into your computer (removing a file is often easier than uninstalling software if you suddenly regret an installation)

However, containers may also have some drawbacks:

  • Can be used to hide away software installation problems and thereby discourage good software development practices.

  • Instead of “works on my machine” problem: “works only in this container” problem?

  • They can be difficult to modify (this is the focus of an exercise below)

  • Container images can become large


Use only official and trusted images! Not all images can be trusted! There have been examples of contaminated images so investigate before using images blindly. Apply same caution as installing software packages from untrusted package repositories.

Where can one share or find images?


Containers-1: Time travel

Scenario: A researcher has written and published their research code which requires a number of libraries and system dependencies. They ran their code on a Linux computer (Ubuntu). One very nice thing they did was to publish also a container image with all dependencies included, as well as the definition file (below) to create the container image.

Now we travel 3 years into the future and want to reuse their work and adapt it for our data. The container registry where they uploaded the container image however no longer exists. But luckily we still have the definition file (below)! From this we should be able to create a new container image.

  • Can you anticipate problems using the definitions file 3 years after its creation? Which possible problems can you point out?

  • Discuss possible take-aways for creating more reusable containers.

 1Bootstrap: docker
 2From: ubuntu:latest
 5    # Set environment variables
 6    export VIRTUAL_ENV=/app/venv
 8    # Install system dependencies and Python 3
 9    apt-get update && \
10    apt-get install -y --no-install-recommends \
11        gcc \
12        libgomp1 \
13        python3 \
14        python3-venv \
15        python3-distutils \
16        python3-pip && \
17    apt-get clean && \
18    rm -rf /var/lib/apt/lists/*
20    # Set up the virtual environment
21    python3 -m venv $VIRTUAL_ENV
22    . $VIRTUAL_ENV/bin/activate
24    # Install Python libraries
25    pip install --no-cache-dir --upgrade pip && \
26    pip install --no-cache-dir -r /app/requirements.txt
29    # Copy project files
30    ./requirements.txt /app/requirements.txt
31    ./ /app/
32    # Copy data
33    /home/myself/data /app/data
34    # Workaround to fix dependency on fancylib
35    /home/myself/fancylib /usr/lib/fancylib
38    # Set the environment variables
39    export LANG=C.UTF-8 LC_ALL=C.UTF-8
40    export VIRTUAL_ENV=/app/venv
43    # Activate the virtual environment
44    . $VIRTUAL_ENV/bin/activate
45    # Run the application
46    python /app/

(optional) Containers-2: Explore two really useful Docker images

You can try the below if you have Docker installed. If you have Singularity/Apptainer and not Docker, the goal of the exercise can be to run the Docker containers through Singularity/Apptainer.

  1. Run a specific version of Rstudio:

    $ docker run --rm -p 8787:8787 -e PASSWORD=yourpasswordhere rocker/rstudio

    Then open your browser to http://localhost:8787 with login rstudio and password “yourpasswordhere” used in the previous command.

    If you want to try an older version you can check the tags at and run for example:

    $ docker run --rm -p 8787:8787 -e PASSWORD=yourpasswordhere rocker/rstudio:3.3
  2. Run a specific version of Anaconda3 from

    $ docker run -i -t continuumio/anaconda3 /bin/bash

Resources for further learning