List of exercises

Full list

This is a list of all exercises and solutions in this lesson, mainly as a reference for helpers and instructors. This list is automatically generated from all of the other pages in the lesson. Any single teaching event will probably cover only a subset of these, depending on their interests.

Motivation and wishlist

In wishlist.md:

In wishlist.md:

In-code documentation

In in-code-documentation.md:

In-code-1: Comments

Let’s take a look at two example comments (comments in python start with #):

Comment A

# Now we check if temperature is larger then -50:
if temperature > -50:
    print('do something')

Comment B

# We regard temperatures below -50 degrees as measurement errors
if temperature > -50:
    print('do something')

Which of these comments is best? Can you explain why?

In in-code-documentation.md:

In in-code-documentation.md:

In-code-2: add in-code documentation

Update this code snippet so it is well-documented:

import pandas as pd

def x(a, print_columns=False):
   b = pd.read_excel(a)
   column_headers = list(b.columns.values)
   if print_columns:
       print("\n".join(column_headers))
   return column_headers

In in-code-documentation.md:

Writing good README files

In writing-readme-files.md:

Exercise README-1: Write a README file for an example Python project

The example project

Here’s the example project. For this project we transformed the code snippets from the previous episode into a single script analyse_spreadsheet.py

Let’s take a look at the script. You don’t need to understand the script completely, all you need to know is:

  • The functions mean_temperature and get_spreadsheet_columns from previous episode are in there.

  • We added a main function that is called when you run the script (you could run this python script by calling python analyse_spreadsheet.py on the command line). It will prompt the user for a file name, print the columns in the spreadsheet, and print the mean temperature.

That’s all there is to this project! (You can ignore the other files in the repository, we’ll get back to them in episode 4)

The exercise

  1. Fork the the example project to your own github namespace

  2. Add a file called README.md (you can use the github web interface or work locally (i.e. git clone, edit the file, git add, git commit, git push))

  3. Add some content to your README file. Think about what you want the audience to know about your project! It does not matter whether the information is correct, it is more important that you have all the components that make up a good README file.

  4. Note that the README file is nicely rendered on the github repository page.

  5. Compare your README file with that of others, is all the essential information in there? NB: The README.md file is written in ‘Markdown’ a very popular lightweight markup language, all you need to know for now is this syntax:

# A section title
## A subsection title
Normal text

A list with items
- item1
- item2

(Optional): Use https://hemingwayapp.com/ to analyse your README file and make your writing bold and clear!

In writing-readme-files.md:

In writing-readme-files.md:

Exercise README-2: Draft or review a README for one of your recent projects

Try to draft a brief README or review a README which you have written for one of your projects.

You can do that either by screensharing and discussing or working individually.

Please note observations and recommendations in the collaborative notes.

In writing-readme-files.md:

Exercise README-3: Review and discuss a README of a project that you have used

In this exercise we will review and discuss a README of a project which you have used. You can also review a library which is popular in your domain of research and discuss their README.

You can do that either by screensharing and discussing or working individually.

When discussing other people’s projects please remember to be respectful and constructive. The goal of this exercise is not to criticize other projects but to learn from other projects and to collect the aspects that you enjoyed finding in a README and to also collect aspects which you have searched for but which are sometimes missing.

Please note observations and recommendations in the collaborative notes.

Sphinx and Markdown

In sphinx.md:

Sphinx-1: Generate the basic documentation template

Create a directory for the example documentation, step into it, and inside generate the basic documentation template:

$ mkdir doc-example
$ cd doc-example
$ sphinx-quickstart

The quickstart utility will ask you some questions. For this exercise, you can go with the default answers except to specify a project name, author name, and project release:

Separate source and build directories (y/n) [n]: <hit enter>
Project name: <your project name>
Author name(s): <your name>
Project release [0.1]: 0.1
Project language [en]: <hit enter>

A couple of files and directories are created:

File/directory

Contents

conf.py

Documentation configuration file

index.rst

Main file in Sphinx

_build/

Directory where docs are built (you can decide the name)

_templates/

Your own HTML templates

_static/

Static files (images, styles, etc.) copied to output directory on build

Makefile

Makefile to build documentation using make

make.bat

Makefile to build documentation using make (Windows)

Makefile and make.bat (for Windows) are build scripts that wrap the sphinx commands, but we will be doing it explicitly.

Let’s have a look at the index.rst file, which is the main file of your documentation:

.. myproject documentation master file, created by
   sphinx-quickstart on Tue May 11 18:38:22 2021.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to myproject's documentation!
=====================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:



Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

  • We will not use the Indices and tables section now, so remove it and everything below.

  • The top four lines, starting with .., are a comment.

  • The next lines are the table of contents. We can add content below:

.. toctree::
  :maxdepth: 2
  :caption: Contents:

  feature-a.md

A common gotcha with directives is that the first line of the content must be indented to the same level as the options (i.e., :maxdepth:).

We now need to tell Sphinx to use markdown files. To do this, we open conf.py and replace the lines

extensions = [
]

with

extensions = ['myst_parser']

source_suffix = ['.rst', '.md']

The first part tells Sphinx to use an extension to parse Markdown files and the second part tells it to actually look for those files.

Let’s create the file feature-a.md which we have just added to index.rst (note that one is .rst and one is .md):

# Some Feature A

## Subsection

Exciting documentation in here.
Let's make a list (empty surrounding lines required):

- item 1

  - nested item 1
  - nested item 2

- item 2
- item 3

We now build the site:

$ ls

_build  _static  _templates  conf.py  feature-a.md  index.rst

$ sphinx-build . _build

Running Sphinx v3.5.4
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 1 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

$ ls _build

_sources  _static  feature-a.html  genindex.html  index.html  objects.inv search.html  searchindex.js

Callout

If you are not using the CodeRefinery Conda environment, you may see the error

Extension error:
Could not import extension myst_parser (exception: No module named 'myst_parser')

To fix this follow the instructions for installing the conda environment.

Now open the file _build/index.html in your browser by:

Linux users type:

$ xdg-open _build/index.html

macOS users type:

$ open _build/index.html

Windows users type:

$ start _build/index.html

Others:

enter file:///home/user/doc-example/_build/index.html in your browser (adapting the path to your case).

Hopefully you can now see a website. If so, then you are able to build Sphinx pages locally. This is useful to check how things look before pushing changes to GitHub or elsewhere.

Note that you can change the styling by editing conf.py and changing the value html_theme (for instance you can set it to sphinx_rtd_theme to have the Read the Docs look).

In sphinx.md:

Sphinx-2: Add content to your example documentation

  1. Add a entry below feature-a.md labeled feature-b.md to the index.rst file.

  2. Create a file feature-b.md in the same directory as your feature-a.md file.

  3. Add some content to feature-b, rebuild with sphinx-build, and refresh the browser to look at the results (Help).

Experiment with the following Markdown syntax:

  • *Emphasized text* and **bold text**

  • Headings

# Level 1

## Level 2

### Level 3

#### Level 4

  • An image: ![](image.png)

  • [A link](http://www.google.com)

  • Numbered lists (numbers adjusted automatically)

1. item 1
2. item 2
3. item 3
1. item 4
1. item 5

  • Simple tables

| No.  |  Prime |
| ---- | ------ |
| 1    |  No    |
| 2    |  Yes   |
| 3    |  Yes   |
| 4    |  No    |

  • Code block

The following is a code block:
```
  def hello():
      print("Hello world")
```

  • Code block specifying syntax highlighting for other language than Python

 ```c

  #include <stdio.h>
  int main()
  {
      printf("Hello, World!");
      return 0;
  }
 ```

  • You could include the contents of an external file using {include} directive, as follows:

 ```{include} ../README.md
 ```

  • It is possible to combine {include} with code highlighting, line numbering, and even line highlighting.

  • We can also use jupyter notebooks (*.ipynb) with sphinx. It requires nbsphinx extension to be installed. See nbsphinx documentation for more information

.. toctree::
  :maxdepth: 2
  :caption: Contents:

  feature-a.md
  <python_notebook_name>.ipynb

In sphinx.md:

(optional) Sphinx-3: Rendering (LaTeX) math equations

There are two different ways to display mathematical equations within Sphinx: pngmath and MathJax. While pngmath displays an equation as an image, MathJax is using scalable vector graphics (quality remains the same after zooming). For this reason, we strongly encourage you to use MathJax for your mathematical equations.

To enable MathJax in Sphinx, you need first to add sphinx.ext.mathjax to the list of extensions in conf.py:

extensions = ['myst_parser', 'sphinx.ext.mathjax']

The following shows how to inline mathematics within a text:

This is an inline equation embedded {math}`a^2 + b^2 = c^2` in text.

This is an inline equation embedded \(a^2 + b^2 = c^2\) in text.

This creates an equation or it’s own line:

```{math}
a^2 + b^2 = c^2
```
\[a^2 + b^2 = c^2\]

For more math syntax (separate to what is above, not needed for this exercise), check the MyST documentation.

Deploying Sphinx documentation to GitHub Pages

In gh_workflow.md:

gh-pages-1: Deploy Sphinx documentation to GitHub Pages

In this exercise we will create an example repository on GitHub and deploy it to GitHub Pages. The example project contains a script for counting the frequency distribution of words in a given file and some documentation generated using Sphinx. For bigger projects, we can have more source files.

Step 1: Go to the documentation-example project template on GitHub and create a copy to your namespace (“Generate”, since this is a template repository).

Clone the repository

The repository contains following two folders, among few other files:

  • source folder contains the source code

  • doc folder contains the Sphinx documentation

The doc folder contains the Sphinx configuration file (conf.py) and the index file (index.rst) and some contents (Markdown files). The conf.py file has been adjusted to be able to autogenerate documentation from sources.

Build HTML pages locally

Inside the cloned repository, build the documentation and verify the result in your browser:

$ sphinx-build doc _build

Test HTML pages links

Inside the cloned repository, check the integrity of all internal and external links:

$ sphinx-build doc -W -b linkcheck -d _build/doctrees _build/html

Step 2: Add the GitHub Action

  • Create a new file at .github/workflows/documentation.yaml with the contents

name: Docs
on: [push, pull_request, workflow_dispatch]
permissions:
    contents: write
jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v3
      - name: Install dependencies
        run: |
          pip install sphinx sphinx_rtd_theme
      - name: Sphinx build
        run: |
          sphinx-build doc _build
      - name: Deploy
        uses: peaceiris/actions-gh-pages@v3
        if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
        with:
          publish_branch: gh-pages
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: _build/
          force_orphan: true

You don’t need to understand all of the above, but you might spot familiar commands in the run: sections.

  • Add, commit and push to GitHub

  • Check the action at https://github.com/<myuser>/documentation-example/actions. Replace <myuser> with your GitHub username.

Step 2: Enable GitHub Pages

  • Go to https://github.com/<myuser>/documentation-example/settings/pages

  • In the “Source” section, choose “Deploy form a branch” in the dropdown menu

  • In the “Branch” section choose “gh-pages” and “/root” in the dropdown menus and click save

  • (You should be able to verify the pages deployment in the Actions list)

Verify the result

That’s it! Your site should now be live on https://<myuser>.github.io/documentation-example/ (replace username).

Verify refreshing the documentation

Finally, make some changes to your documentation

  • Add documentation related to other functions

  • Prerequisites and how to use the program

  • Rules for contribution

  • Some example results (figures, tables, …)

  • Commit and push them, and verify that the documentation website refreshes after your changes (can take few seconds or a minute)

Hosting websites/homepages on GitHub Pages

In gh-pages.md:

gh-pages-2: Host your own github page

  • Deploy own website reusing a template:

    • Follow the steps from GitHub Pages https://pages.github.com/. The documentation there is very good so there is no need for us to duplicate the screenshots

    • Select “Project site”

    • Select “Choose a theme” (for instance “Minimal”)

    • Click “Select theme”

    • Adjust the README.md and commit

    • Browse your page on http://username.github.io/repository (adjust “username” and “repository”)

  • Make a change to the repository after the webpage has been deployed for the first time

  • Please wait few minutes and then verify that the change shows up on the website