List of exercises
Full list
This is a list of all exercises and solutions in this lesson, mainly as a reference for helpers and instructors. This list is automatically generated from all of the other pages in the lesson. Any single teaching event will probably cover only a subset of these, depending on their interests.
Motivation and wishlist
Our motivation (but let us brainstorm first)
You will probably use your code in the future and may forget details.
You may want others to use your code (almost impossible without documentation).
You may want others to contribute to the code.
Shield your limited time and let the documentation answer FAQs.
Our wishlist (but let us brainstorm first)
Versions
Your code project should be versioned (version control).
Enable reproducibility and avoid confusion: documentation should be versioned as well.
Have you ever seen: “We will soon release a new version and are updating the documentation. Some features may not be available in the version you have downloaded.”?
Documentation should be placed and tracked close to the source code
Documenting close to the source code (e.g. subdirectory
doc/
) minimizes barrier to contribute.I should not need to log in to another machine or service and jump through hoops to contribute.
It is often good enough to have a
README.md
orREADME.rst
along with your code/script.
Use a standard markup language
Markup
Markup is a set of human readable instructions that is used to tell the computer how a document shall be styled and structured. By using a markup language we can for example write a *
or -
where we want a bullet point to appear in the rendered document.
offers formatting flexibility, enforces a basic document structure and the rendered documents can be exported to other formats (e.g. for printing). Also, the source can be read by humans without knowledge of the language in case the rendered document is unavailable.
We suggest to use either reStructuredText (RST) or Markdown markup.
GitHub and GitLab automatically render
README.md
orREADME.rst
files.
Copy-paste-able
PDF alone is not enough since copy-pasting out of a PDF document can be difficult.
It is OK to provide a generated PDF in addition to a copy-paste-able format.
Written by humans
Automatically generated documentation (e.g. API documentation) is useful as complementary documentation but it does not replace tutorials written by humans.
Installation instructions
Give step by step instructions for the basic case. Additional information and caveats can be linked from there.
List requirements and dependencies (libraries, compilers, environment).
Include instructions for how to test for correctness after installation.
Make the license explicit
Include a LICENSE file with your source code.
Without a license, your work is under exclusive copyright by default: others are not allowed to re-use or modify anything.
GitHub and GitLab allows to choose a license from common license templates.
Information for contributors
Make it easy for others to contribute: document how you prefer others to contribute.
Users of your code may be shy to contribute code. Your documentation provides a platform for your first contributions.
In-code documentation
Solution
Comment A describes what happens in this piece of code, whereas comment B describes why this piece of code is there, i.e. its purpose. Comments in the form of B are much more useful, comments of form A are redundant and we should avoid them.
In-code-2: add in-code documentation
Update this code snippet so it is well-documented:
import pandas as pd
def x(a, print_columns=False):
b = pd.read_excel(a)
column_headers = list(b.columns.values)
if print_columns:
print("\n".join(column_headers))
return column_headers
Solution
import pandas as pd
def get_spreadsheet_columns(file_loc, print_columns=False):
"""Gets and prints the spreadsheet's header columns
Args:
file_loc (str): The file location of the spreadsheet
print_columns (bool, optional) : A flag used to print the columns to the console (default is False)
Returns:
a list of strings used that are the header columns
"""
file_data = pd.read_excel(file_loc)
column_headers = list(file_data.columns.values)
if print_columns:
print("\n".join(column_headers))
return column_headers
Writing good README files
Exercise README-1: Write a README file for an example Python project
The example project
Here’s the example project. For this project we transformed the code snippets from the previous episode into a single script analyse_spreadsheet.py
Let’s take a look at the script. You don’t need to understand the script completely, all you need to know is:
The functions
mean_temperature
andget_spreadsheet_columns
from previous episode are in there.We added a
main
function that is called when you run the script (you could run this python script by callingpython analyse_spreadsheet.py
on the command line). It will prompt the user for a file name, print the columns in the spreadsheet, and print the mean temperature.
That’s all there is to this project! (You can ignore the other files in the repository, we’ll get back to them in episode 4)
The exercise
Fork the the example project to your own github namespace
Add a file called
README.md
(you can use the github web interface or work locally (i.e.git clone
, edit the file,git add
,git commit
,git push
))Add some content to your README file. Think about what you want the audience to know about your project! It does not matter whether the information is correct, it is more important that you have all the components that make up a good README file.
Note that the README file is nicely rendered on the github repository page.
Compare your README file with that of others, is all the essential information in there? NB: The
README.md
file is written in ‘Markdown’ a very popular lightweight markup language, all you need to know for now is this syntax:
# A section title
## A subsection title
Normal text
A list with items
- item1
- item2
(Optional): Use https://hemingwayapp.com/ to analyse your README file and make your writing bold and clear!
Solution
A README file for this project could look like this:
# Temperature analysis in spreadsheets
A python script for the analysis of temperatures in excel files.
## Why should I use this project ?
It makes it easy to analyse excel files with temperatures in them.
## Setup
You need `python>3.5` to run this script.
The project depends on the `pandas` library, install it with pip:
`pip install pandas`
## How to run?
You can run the script from the command-line using
```
python analyse_spreadsheet.py
```
You can use functions directly, for example: calculate the mean temperature of some data:
```python
from analyse_spreadsheet import mean_temperature
print(mean_temperature(data))
```
## How to cite this project?
Please email `training@esciencecenter.nl` to get instructions on how to properly cite this project.
## Contributing
You are welcome to contribute to the code via pull requests. Please have a
look at the [NLeSC
guide](https://nlesc.gitbooks.io/guide/content/software/software_overview.html)
for guidelines about software development.
Exercise README-2: Draft or review a README for one of your recent projects
Try to draft a brief README or review a README which you have written for one of your projects.
You can do that either by screensharing and discussing or working individually.
Please note observations and recommendations in the collaborative notes.
Exercise README-3: Review and discuss a README of a project that you have used
In this exercise we will review and discuss a README of a project which you have used. You can also review a library which is popular in your domain of research and discuss their README.
You can do that either by screensharing and discussing or working individually.
When discussing other people’s projects please remember to be respectful and constructive. The goal of this exercise is not to criticize other projects but to learn from other projects and to collect the aspects that you enjoyed finding in a README and to also collect aspects which you have searched for but which are sometimes missing.
Please note observations and recommendations in the collaborative notes.
Sphinx and Markdown
Sphinx-1: Generate the basic documentation template
Create a directory for the example documentation, step into it, and inside generate the basic documentation template:
$ mkdir doc-example
$ cd doc-example
$ sphinx-quickstart
The quickstart utility will ask you some questions. For this exercise, you can go with the default answers except to specify a project name, author name, and project release:
Separate source and build directories (y/n) [n]: <hit enter>
Project name: <your project name>
Author name(s): <your name>
Project release [0.1]: 0.1
Project language [en]: <hit enter>
A couple of files and directories are created:
File/directory |
Contents |
---|---|
conf.py |
Documentation configuration file |
index.rst |
Main file in Sphinx |
_build/ |
Directory where docs are built (you can decide the name) |
_templates/ |
Your own HTML templates |
_static/ |
Static files (images, styles, etc.) copied to output directory on build |
Makefile |
Makefile to build documentation using make |
make.bat |
Makefile to build documentation using make (Windows) |
Makefile
and make.bat
(for Windows) are build scripts that wrap the sphinx commands, but
we will be doing it explicitly.
Let’s have a look at the index.rst
file, which is the main file of your documentation:
.. myproject documentation master file, created by
sphinx-quickstart on Tue May 11 18:38:22 2021.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to myproject's documentation!
=====================================
.. toctree::
:maxdepth: 2
:caption: Contents:
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
We will not use the
Indices and tables
section now, so remove it and everything below.The top four lines, starting with
..
, are a comment.The next lines are the table of contents. We can add content below:
.. toctree::
:maxdepth: 2
:caption: Contents:
feature-a.md
A common gotcha with directives is that the first line of the content must be indented to the same level as the options (i.e., :maxdepth:).
We now need to tell Sphinx to use markdown files. To do this, we open
conf.py
and replace the lines
extensions = [
]
with
extensions = ['myst_parser']
source_suffix = ['.rst', '.md']
The first part tells Sphinx to use an extension to parse Markdown files and the second part tells it to actually look for those files.
Let’s create the file feature-a.md
which we have just added to
index.rst
(note that one is .rst
and one is .md
):
# Some Feature A
## Subsection
Exciting documentation in here.
Let's make a list (empty surrounding lines required):
- item 1
- nested item 1
- nested item 2
- item 2
- item 3
We now build the site:
$ ls
_build _static _templates conf.py feature-a.md index.rst
$ sphinx-build . _build
Running Sphinx v3.5.4
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 1 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.
$ ls _build
_sources _static feature-a.html genindex.html index.html objects.inv search.html searchindex.js
Callout
If you are not using the CodeRefinery Conda environment, you may see the error
Extension error:
Could not import extension myst_parser (exception: No module named 'myst_parser')
To fix this follow the instructions for installing the conda environment.
Now open the file _build/index.html
in your browser by:
Linux users type:
$ xdg-open _build/index.html
macOS users type:
$ open _build/index.html
Windows users type:
$ start _build/index.html
Others:
enter file:///home/user/doc-example/_build/index.html
in your browser (adapting the path to your case).
Hopefully you can now see a website. If so, then you are able to build Sphinx pages locally. This is useful to check how things look before pushing changes to GitHub or elsewhere.
Note that you can change the styling by editing conf.py
and changing the value html_theme
(for instance you can set it to sphinx_rtd_theme
to have the Read the Docs look).
Sphinx-2: Add content to your example documentation
Add a entry below feature-a.md labeled feature-b.md to the
index.rst
file.Create a file
feature-b.md
in the same directory as yourfeature-a.md
file.Add some content to feature-b, rebuild with
sphinx-build
, and refresh the browser to look at the results (Help).
Experiment with the following Markdown syntax:
*Emphasized text* and **bold text**
Headings
# Level 1
## Level 2
### Level 3
#### Level 4
An image:

[A link](http://www.google.com)
Numbered lists (numbers adjusted automatically)
1. item 1
2. item 2
3. item 3
1. item 4
1. item 5
Simple tables
| No. | Prime |
| ---- | ------ |
| 1 | No |
| 2 | Yes |
| 3 | Yes |
| 4 | No |
Code block
The following is a code block:
```
def hello():
print("Hello world")
```
Code block specifying syntax highlighting for other language than Python
```c
#include <stdio.h>
int main()
{
printf("Hello, World!");
return 0;
}
```
You could include the contents of an external file using
{include}
directive, as follows:
```{include} ../README.md
```
It is possible to combine
{include}
with code highlighting, line numbering, and even line highlighting.We can also use jupyter notebooks (*.ipynb) with sphinx. It requires
nbsphinx
extension to be installed. See nbsphinx documentation for more information
.. toctree::
:maxdepth: 2
:caption: Contents:
feature-a.md
<python_notebook_name>.ipynb
(optional) Sphinx-3: Rendering (LaTeX) math equations
There are two different ways to display mathematical equations within Sphinx:
pngmath
and MathJax
. While pngmath
displays an equation as an image,
MathJax
is using scalable vector graphics (quality remains the same after
zooming). For this reason, we strongly encourage you to use MathJax
for
your mathematical equations.
To enable MathJax
in Sphinx, you need first to add sphinx.ext.mathjax
to
the list of extensions in conf.py
:
extensions = ['myst_parser', 'sphinx.ext.mathjax']
The following shows how to inline mathematics within a text:
This is an inline equation embedded {math}`a^2 + b^2 = c^2` in text.
This is an inline equation embedded \(a^2 + b^2 = c^2\) in text.
This creates an equation or it’s own line:
```{math}
a^2 + b^2 = c^2
```
For more math syntax (separate to what is above, not needed for this exercise), check the MyST documentation.
Deploying Sphinx documentation to GitHub Pages
gh-pages-1: Deploy Sphinx documentation to GitHub Pages
In this exercise we will create an example repository on GitHub and deploy it to GitHub Pages. The example project contains a script for counting the frequency distribution of words in a given file and some documentation generated using Sphinx. For bigger projects, we can have more source files.
Step 1: Go to the documentation-example project template on GitHub and create a copy to your namespace (“Generate”, since this is a template repository).
Clone the repository
The repository contains following two folders, among few other files:
source folder contains the source code
doc folder contains the Sphinx documentation
The doc folder contains the Sphinx configuration file (conf.py
) and the index file (index.rst
) and some contents (Markdown files).
The conf.py
file has been adjusted to be able to autogenerate documentation from sources.
Build HTML pages locally
Inside the cloned repository, build the documentation and verify the result in your browser:
$ sphinx-build doc _build
Test HTML pages links
Inside the cloned repository, check the integrity of all internal and external links:
$ sphinx-build doc -W -b linkcheck -d _build/doctrees _build/html
Step 2: Add the GitHub Action
Create a new file at
.github/workflows/documentation.yaml
with the contents
name: Docs
on: [push, pull_request, workflow_dispatch]
permissions:
contents: write
jobs:
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
- name: Install dependencies
run: |
pip install sphinx sphinx_rtd_theme
- name: Sphinx build
run: |
sphinx-build doc _build
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
with:
publish_branch: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/
force_orphan: true
You don’t need to understand all of the above, but you
might spot familiar commands in the run:
sections.
Add, commit and push to GitHub
Check the action at
https://github.com/<myuser>/documentation-example/actions
. Replace<myuser>
with your GitHub username.
Step 2: Enable GitHub Pages
Go to
https://github.com/<myuser>/documentation-example/settings/pages
In the “Source” section, choose “Deploy form a branch” in the dropdown menu
In the “Branch” section choose “gh-pages” and “/root” in the dropdown menus and click save
(You should be able to verify the pages deployment in the Actions list)
Verify the result
That’s it! Your site should now be live on
https://<myuser>.github.io/documentation-example/
(replace username).
Verify refreshing the documentation
Finally, make some changes to your documentation
Add documentation related to other functions
Prerequisites and how to use the program
Rules for contribution
Some example results (figures, tables, …)
Commit and push them, and verify that the documentation website refreshes after your changes (can take few seconds or a minute)
Hosting websites/homepages on GitHub Pages
gh-pages-2: Host your own github page
Deploy own website reusing a template:
Follow the steps from GitHub Pages https://pages.github.com/. The documentation there is very good so there is no need for us to duplicate the screenshots
Select “Project site”
Select “Choose a theme” (for instance “Minimal”)
Click “Select theme”
Adjust the README.md and commit
Browse your page on
http://username.github.io/repository
(adjust “username” and “repository”)
Make a change to the repository after the webpage has been deployed for the first time
Please wait few minutes and then verify that the change shows up on the website
In-code-1: Comments
Let’s take a look at two example comments (comments in python start with
#
):Comment A
Comment B
Which of these comments is best? Can you explain why?