Questions and notes from workshop day 5

Icebreaker questions

Icebreaker Q1: What tools do you use for writing code?

Icebreaker Q2: What tools do you use for debugging code?

Icebreaker Q3: Are you already using Jupyter for some projects?

Questions

  1. Is there a link on explanation on how to install jupyter on git bash?

    • Usually it would be done through conda. For the purposes of this course, run it through the coderefinery anaconda environment.
    • Can I do these commands in CR anaconda?
        mkdir  jupyterlab-demo
    cd jupyterlab-demo
    jupyter-lab

Jupyter

https://coderefinery.github.io/jupyter/

  1. Is there an easy way run the notebook as a Python script through the Python interpreter "outside" of JupyterLab?

    • Usually, you would export the code to standard Python. What is in the cells is standard Python, you mainly need to put it in plain text form to be executable.
    • By this question, did you mean to run the notebook as a Python script through the Python interpreter "outside" of JupyterLab?
      • That's right, this is what I meant. I have rephrased the question :-)

  2. Is there a way to conveniently integrate R, Python, and documentation into one notebook?

    • The rpy2 library may offer a solution. More info [here]((https://pypi.org/project/rpy2/). This blog may be useful as well.
      • What about running Python within R (using an R Markdown notebook)? Is that possible?

  3. Should instructor #1's sound be softer still? (+1)

    • Thanks! I will relay to instructors ...
    • Yes, the sound is too loud compared to [name=Teemu's] in our video conference room...
    • And someone is spinning a spinner? There is some background noise
    • Thank you, that solved the issue
    • Now it is just fine!(+1)
    • Thank you for trying to solve it - even if it interrupts, it is still nice for a full day of listening that it is more balanced!

Notebook interface

https://coderefinery.github.io/jupyter/interface/

  1. Do notebooks offer variable viewer, like Matlab or Spyder?

    • Is this for debugging? Like a variable inspector?
    • Yes, they do and I've seen it. It might be an extension or built in ... I don't remember how to get to it...
    • There is also this one: https://github.com/lckr/jupyterlab-variableInspector but I haven't used it yet, only watched the demo.

  2. How do these components look like in Google Colab?

    • Pretty similar in concept but I think they have re-done a lot of it behind the scenes.

  3. Code along? Or not?

    • Demonstration, watch only.

  4. What's the difference between jupyter-lab and jupyter-notebook?

    • jupyter-notebook was there first and later jupyter-lab got developed, which can do a bit more than only notebooks: easier navigation, several notebooks opened at the same time, tabs, terminal, extensions, plugins.

  5. Markup vs markdown?

    • markdown: one specific syntax (actually a whole group of related but different ones)
    • markup: general term for adding in extra syntax markers to text that affect display. There are many different types of markup syntaxes

  6. What does (*) mean near Control button in the shortcuts?

    • Is that the symbol for the corresponding key on a Mac keyboard? (Linux person answering here so I am not sure ...)
    • Yes, it's the command key.

A first computational notebook

https://coderefinery.github.io/jupyter/first-notebook/

Exercices until xx:55

https://coderefinery.github.io/jupyter/first-notebook/#an-example-computational-notebook

I am:

  1. What is an Iframe?

    • iframe: it's embedding one web page into another webpage. In this case we are saying "embed this other web page in notebook so that you can refer to it more easily".
      • we need to run this command in the jupyter-lab, you mean?
        • oh yeah , cool it works

  2. I have copied the ## relevant formulas into the cell and ran it with shift-enter, but it gives me a syntax error. How do I let notebook know that it is all markdown?

    • What's the last line(s) of the error message? line before? 
      •   Cell In[3], line 3`
  square area: s = (2 r)^2$
  SyntaxError: invalid syntax
      • You are likely running the cell as a code cell. Change it to a Markdown cell. In the menu bar, where it says "Code", click, dropdown, select "Markdown".
        • Ah yes, got it, thanks!

  3. What's the potential advantage of opening Jupyter notebook from Jupyter lab other than opening it directly by typing jupyter-notebook?

    • jupyter-lab is a different interface for jupyter-notebook. Both read the same file format but lab is a newer interface (e.g. multiple tabs in one window). Is this what you were asking?
      • Yes, thanks!

  4. Yesterday, the instructor (the one with the cat) mentioned you had so much feedback, ~80 <smth> of text. Is <smth> KB or MB? :)

    • It was KB.

  5. jupyter-lab answers: command not found

    • I got it. Needed to do: conda activate coderefinery
      • great :-)

  6. The result is 3.116. Not as accurate as I thought.

    • Does it get better with more points/throws?
      • Yes, it is better, but with 10k points it is still far. :-( Another think I am finding is that after change the value of the points I have to rerun all the code blocks. Is there any way to update all the remaining blocks after when changing the value of points ??
      • Yes. I can't remember the shortcut right now (learner here), but just press the "fast foward" double arrow in the top - it will restart the kernel and rerun all cells (basically deleting all saved variables and running everything from the top again). (with 100k points i get 3.1424 - but will always change between runs, since it is with random generated values. to get a good estimate I would suggest using 1kk, but don't plot that ;))
        • Ok, done. Thanks for the tip. ;-) with 200k 3.1416 exactly. Perfect.

  7. This has bugged me for some time now, but is there a way to make the images centered rather than on the far left?

    • You mean now images in a Markdown cell?
      • Yes!
        • As far as I know, Markdown does not have concepts/means for that (please somebody correct me if you know) but one thing that one can do, is to use html styling inside Markdown but I haven't tried this.
          • Alternative: maybe this one has more styling options: https://jupyterbook.org/
            • Awesome, thank you! :)
              • OK, I think I have tried the html method and i tworks ok. I was wondering if it was possible using Md.

  8. I got an error:

 ModuleNotFoundError                       
 Traceback (most recent call last)
 Cell In[1], line 4
       1 # importing modules that we will need
       3 import random
 ----> 4 import matplotlib.pyplot as plt
 
ModuleNotFoundError: No module named 'matplotlib'

  1. What does it mean in first-notebook, markdown cell? Do Jupyter cells have different type? How to change it?

    • there are two types: code cell for code (Python or other language) and Markdown cell for documentation. one way to switch is above in the notebook tab I see a "Code (arrow)" where I can switch between Code, Markdown, and Raw (not sure what that will be) - did you find it? There is also a keyboard shortcut to toggle ... (don't remember it though)
      • Yes, I found a code button. Thanks!
    • keyboard shortcut to switch between cell types is "m" and "y"
      • I'm not sure about the shortcuts, just press "m" / "y" when in a cell to switch the cell type?
        • You need to have the cell selected but can't be writing in it for the keyboard shortcut to work.
          • Oh, I see now, got it.
            • esc + m gives Markdown, esc + y gives code, esc + enter opens the cell for writing again.

  2. Is there a simple way to look into the docs of functions of libraries inside Jupyter-Lab?

    • In the top menu, select Help -> Show Contextual Help, then place yoursef over a function etc (eg. ax.scatter) and see the docs there. Also you can SHIFT+TAB over a function etc. and a popup with the docs appears.

  3. I have difficulties with shortcuts. All Ctrl-Enter and Shoft-Enter work but the capital letters alone keep writing to cells. How should I run the capital letter shortcuts (Windows 11)?

    • With the cell selected, press ESCAPE to go into command mode. Click on the cell to go back to edit mode. Cells are modal. -THNX!

  4. Since we talk about jupyter notebook today, it will be nice to mention the google colab. I saw that you will talk about how to share the notebook. What are the advantages and disadvantages when we have a large amount of data that are not stored in our computer but remotely?

    • I think we'll mention.
      • But I guess the advanges are like everything with cloud: convenient for getting started, but it's someone elses computer. The conditions can change at any time, less security, more cost.
        • Nice with Colab is the collaboration option. Tried a few years ago, not as real-time as I would have wanted.

  5. What are static notebook and dynamic notebook?

    • Not completely sure what the context was but you can browse a notebook on GitHub but then it is "static" (you can't modify it live and change values) but then you can also run a notebook through Binder and then it is dynamic and looks like if I ran it on my computer and I can add cells and modify values and test how the notebook will adapt.
      • Here is the notebook: https://github.com/coderefinery/jupyter/blob/main/example/darts.ipynb (static version, later we will learn how to share notebooks which are dynamic and can be modified). This was the context just below the exercise.
        • Ah yes, so my answer was then relevant :-) Later we will learn how to share a notebook that others can run AND modify without installing Jupyter on their computer.

  6. How do you abort running a code?

    • Top menu: Kernel -> Interrupr Kernel, or alternatively press the little "stop" square near the "play" icon.

  7. I'm not sure if I understand the use case of Jupyter lab. Can it be used for sensitive data?

    • Since it's running on your own computer, it's as secure as your own computer. Which is one reason for it vs similar cloud services.
    • But more broadly, it's about an easy way to do things interactively. It's not good for everything, but conventient for certain things.

  8. How to properly close the Jupyter notebook session in a browser that was started by jupyter-lab command in a Linux terminal? Close the tab and Ctrl+c in the terminal?

    • To shutdown the JupyterLab session completely: File -> Shut Down. To just close the notebook you are editing just close the tab, but the kernel remains active, to shut it down Kernel -> Shut Down Kernel (or in the Kernel management side panel (black circle with white seuqre inside). If you CTRL+C in the terminal it will also kill the JupyterLab session and the web browser will tell you that can not connect or something similar.

  9. Ctrl+q doesnt work for closing notebooks?

    • CTRL+C kills the whole JupyterLab session.
      • In Git Bash? Or in Jupyter itself?
        • In Git Bash
    • Remember to save your work before that.

  10. How can I use git bash now, while jupyter is running? I cannot type anything in git bash.

    • Can you just open another git-bash terminal?
      • Yes.
        • Then that's the way to go. Alternatively, you could probably try running jupyter in the background, but this would lead to outputs from jupyter being written to that terminal. What you can do (but with quite some effor) is running jupyter as a daemon, but For your local system I would not recommend it, but just have one terminal running it.

Notebooks and version control

https://coderefinery.github.io/jupyter/version-control/ Demo: don't type along yet.

  1. Bigger font in the terminal please! Much better, thanks.

    • Thx for the feedback.

  2. Did you create that yml file?

    • I am guessing it is this file: https://github.com/coderefinery/software/blob/main/environment.yml ? This is the file we install the coderefinery environment from. But I was distracted and did not see how the instructors got to that file.

  3. I don't see the yml file in my folder.

    • It's put there specially, I think from installation instructions, don't worry.
    • I guess it'd be the coderefinery environment file they provided for the setup.
      • Indeed. you don't need it here but I guess the instructor prefers to keep the environment file close to the project because they have many other environment files on the disk. RB: I do this personally the same way.

  4. I got an error trying to run jupyter labextension install @jupyterlab/github:

"An error occurred.
ValueError: Please install Node.js and npm before continuing installation. You may be able to install Node.js from your package manager, from conda, or directly from the Node.js website (https://nodejs.org)."

  1. I read a small piece of text regarding OpenCommit, an AI generator of better commit messages to git. Any thoughts on that?

    • I have not checked yet -MJ
      • This is the link of it
        • Looks cool! I wonder how accurate the commit messages then are. At least they look as "good commit messages" in the screenshot there.
    • I might say "if the message can be generated from the code, maybe it's not useful. More useful is why it's made: the human side. what bug fixed? how found?"
      • Are we humans doing better? Many commit messages are exactly like you described, too vague and really focused on what have changed. Would be nice to have some sort of good practices for commit messages, so we all could improve on that.
        • Some resources on good commit messages: https://initialcommit.com/blog/git-commit-messages-best-practices, https://www.freecodecamp.org/news/writing-good-commit-messages-a-practical-guide/, https://www.gitkraken.com/learn/git/best-practices/git-commit-message

  2. How do I paste something into nano?

    • what OS? It's a question of what terminal you have, try right clicking, middle clicking, or left&right clicking at same time.
      • Windows, right click worked.

  3. How do you get that cool green checkmark or red X for the git folders?

    • try searching "checkmark in shell prompt". It shows the exit status (success/failure) of each command. There are many interesting prompt customizations! Also "git-aware prompt". I'm not sure exactly which one is being used here...
      • Thanks, I'll have a look!

  4. Surprising correspondence between a Wolfram Mathematica notebook and a Jupyter notebook! The same cells concept, raw files in both look in the same way! Even the working documents in both are called in the same way, namely notebook! How it that possible?

    • One got inspired by the other? :-) and that's OK in my view since Jupyter is open source code and is inspiration for a number of similar tools. But I don't know the history here and I am not sure who was first.
    • Mathematica was first, but I agree, nothing wrong with basing on good concepts from other systems.

  5. Could we have a break? (+1)

    • We will raise this. Yes.
    • We will have break in a few minutes.

  6. How can I close the notebook and access the bash terminal again? After I open the notebook, I can't use the terminal to type anymore. Any command I need to know? Even after using ctrl+c, I do not seem to be able to type in the terminal.

    • Try Control-C to quit the Jupyter process.
    • Open another terminal :-) If you ctrl+c you'll kill the JupyterLab session completely.

  7. How do you use git commands inside the jupyter environment?

    • Either in the terminal on jupyterlab or using plugins/extensions (as demonstrated now) where one can stage/commit/... on the side bar.
      • What is the terminal on jupyterlab? And I could not follow the instructions as demonstrated.
        • Is a terminal emulator. In the "Launcher" tab, go to the bottom, in the "Other" section press the "Terminal" icon.
    • in Jupyter you can also run bash commands in a code cell if you start the line with ! eg. !ls.

Break until xx:27

Then we;ll discuss the Binder service before going to documentation.

  1. I tried refreshing the notebook and it does not want to load. It says in the browser below waiting for localhost. Maybe because I was using Git Bash and it lost connection? I still have git bash open with a connection to jupyter-labs. Weird.

    • You likely CTRL+C in the terminal where you ran "jupyter-lab" thus killing your JupyterLab session.
      • I CTRL+C in the terminal just now and funny thing is that it started the connection. I used another terminal to set up git. Maybe thats why?

  2. I found this session very useful so far but the last section was a little fast for me :)

    • Thanks for the feedback.

  3. Sorry, now I'm confused. Will there be an exercise for the section "Notebooks and version control" after the break? Or do we go straight to the next lesson?

    • This was a demonstration. After the break we will have an exercise on how to share notebooks.

  4. Wolfram Mathematica has its own editor to type in commands, perform cells, etc. What about Jupyter, are there only web browser based tabs for editing or is there an app with editor I can install on Fedora/Ubuntu?

    • I'm not aware of any specific editors but you can open/run/edit notebooks in Visual Studio Code (with the right extension). instructions
    • Spyder also has some extensions that allow you to run jupyter notebooks.
    • JupyterLab Desktop – I think also open source so I don't know about the support.
      • Do they all support cell-based view?
        • Don't know about spyder. JupyterLab Desktop looks the same as in web browser and I think VSCode has cells but looks different
          • :+1:

  5. How do I create the .gitignore file in the jupyter lab terminal? I cannot use nano.

    • I think "new text file" (and then renaming to .gitignore) would work. I hope that jupyterlab doesn't hide the files begining with . (hidden files)
    • nano works for me
    • Also in Jupyter (with Git stuff installed) you can in the commit view / staging area just right-click a file and select "add to gitinore"
      • New text file does not work, nano either and I cannot see where I can add to gitignore in the commit view
        • I am curious because all those should work. The issue might be somewhere else. Anyway needs more information to be able to debug.
          • This is what I get:
    (base) C:\Users\20225533\AppData\Local\anaconda3\envs\coderefinery\jupyterlab-demo>nano requirements.txt
'nano' is not recognized as an internal or external command, operable program or batch file.
    • Use another text editor that you have installed on your computer.

  6. After I close my server using ctrl+c, I still cannot type in the terminal. Can you please explain how can I use the same terminal?

    • It might still be running, waiting for some other process. Try using ctrl+c multiple times in a row.
    • It might be asking you if you actually want to shut it down (as in type "yes" or "no"), but the message get's lost in the output.
    • I sometimes have to type ctrl+c multiple times.

Sharing notebooks on Binder, exercise until xx:44

https://coderefinery.github.io/jupyter/sharing/

  1. How do I push the notebook to the git repository? And is it only the notebook or the whole folder I need to push? (+1)
    • The notebook file and requirements.txt need to be pushed
    • How to push:
      • git init
      • git add mynotebook.ipynb
      • git add requirements.txt
      • git commit -m "adding notebook and requirements file"
      • Now either follow instructions from github or:
      • git push GITHUBADDRESS master (assuming the branch name is master)
git push git@github.com:user/githubrepo.git master
Enter passphrase for key '/c/Users/user/.ssh/id_ed25519':
Everything up-to-date

  1. Do we do the git push at anaconda or at git bash? It was not clear.

    • git bash (we need a terminal that knows about git)
    • or: terminal in the jupyterlab should also work

  2. Can you share an example solution once the exercise is over?

    • yes good idea, working on it ...
      • repo from 3 years ago: https://github.com/bast/binder-example
      • let's see whether the rendered Binder still works: https://mybinder.org/v2/gh/bast/binder-example/HEAD
        • testing ... suspense ... reproducibility being stress tested ...
          • drumroll .... it works! a repository I haven't touched in 3 years still runs and does the same thing thanks to requirements.txt and Binderization
          • except an image that was loaded from another web address which moved
    • Thanks!

  3. How do I get away from jupiter lab in the terminal ctrl q is not working?

    • Try "control-c", maybe multiple times quickly. -THANK YOU!

  4. See question 46. how to continue after push?

    • looking ...

  5. I am not able to connect to my repisotory:

"Failed to add the host to the list of known hosts (/u/.ssh/known_hosts).
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository."

The authenticity of host 'github.com (<ip-address>)' can't be established.
ED25519 key fingerprint is SHA256:...
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Failed to add the host to the list of known hosts (/u/.ssh/known_hosts).
git@github.com: Permission denied (publickey).

  1. I am confused of how we managed to add the jupyter notebook in GitHub. I undestand that we need to clone etc, but then when I did git add "jupyter notebook", I get the error message fatal: pathspec 'calculating_pi.ipynb' did not match any files.

    • That error indicates that the file has either a different name or is perhaps not in the same folder?
      • The name is correct but in anaconda I had a diferent path directory than the one in git bash. Is it possible to share a link with instructions probably? Solved :) I found the jupyter notebook in git bash and I saved in the directory connected to GitHub.
        • Well done!

  2. Can I use the Binder integration on GitLab too?

    • Yes, GitLab, Zenodo, figshare, dataverse, ... the Git repository can be in principle anywhere if it is publicly accessible. But it is a good idea to put it into a place that won't disappear in 2 years.

  3. Binder gives an error - saying that I should check the URL, no idea where the error is..

    • It was set to private on GitHub... whoops...

  4. I cannot do a pull request to merge master into main, why? It just says 'there is nothing to compare' (but there clearly is, if you check out the master branch vs. the main branch) https://github.com/user/BinderSharingDemo.git

    • The two branches look unrelated: https://github.com/user/BinderSharingDemo/network.
    • It looks to me like main was created by GitHub and then master was pushed from a different local git history.
      • Yes, so what? I don't know how else I would have done it.
    • In this case GitHub does not know how to relate them during a pull request. You can typically either push a master/main branch from local to GitHub or to create it on GitHub and clone it to computer but here you did both :-) recovery: I would on GitHub use "Add file" button to add the files to the main branch directly in the browser and then I would delete the master branch. There are also command line options to recover if you are interested. - Aha, so if I push the master from git bash to GitHub it makes a new branch because in GitHub the branch is called main? How do i push the master branch from git bash to the main branch in github? - Rename the master brach to main: git branch -m main (being in the master branch of course) - Yes, thanks! This is very useful and kind of unlogic that GitHub is called main`` and git bash is called master by default. - I configure also my local git to create new repos with main as branch to avoid this mixup. search for "defaultBranch" on the web. - I tried pushing again after renaming the branch to main. Now it gives me an error:
    ! [rejected]        main -> main (fetch first)
    error: failed to push some refs to 'github.com:pvand/Share_exercise.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
  1. Complete failure - I was able to set up repos and push fine the other day.
git push -u origin main
To github.com:user/darts-demo.git
 ! [rejected]        main -> main (fetch first)
error: failed to push some refs to 'github.com:user/darts-demo.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

  1. I get the error in binder Could not resolve ref for gh:username/coderefinery/HEAD. Double check your URL. GitHub recently changed default branches from "master" to "main". Is it because my repo is private?

    • Ah, yes. If it is private, then binder can't see it and find it.

  2. How we do get the conda enviroment file into the file repository? Will this be used when running the notebook through binder?

    • Y,es instead of requirements.txt you can add environment.yml and Binder will use it. Example of this: https://github.com/coderefinery/software
      • Where can I get the environment.yml from?
        • You mean this one? https://github.com/coderefinery/software/blob/main/environment.yml
        • Yes, thank you! I thought I could find it on my system somewhere once I already the conda environment installed. Do you have any courses on conda environemnts in coderefinery that I could use to learn a bit more?
          • This is a good one: https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/ (not from us)
          • Awesome, thank you!

Sharing dynamic notebooks on Binder

https://coderefinery.github.io/jupyter/sharing/#sharing-dynamic-notebooks-on-binder

I am:

Wrap-up Q&A.

Notebooks in other languages

https://coderefinery.github.io/jupyter/first-notebook/#notebooks-in-other-languages

Documentation

https://coderefinery.github.io/documentation/

Discussion

Why is project documentation important? Why?

How would you describe a useful documentation?

How can you motivate your colleagues to contribute to the documentation?

Question (continued)

  1. Does environment.yml do the same as requirements.txt (but then using conda instead of pip)? In other words, can you opt for the former instead of the latter?
    • for the coderefinery environment, or in general?
    • for coderefinery enviroment: yes, in principle. we haven't tested requirements.txt so much
    • in general: they do roughly the same thing, but environment.yml can include much more software (like support libraries), so the environments are more self-contained.

Break until xx:10

  1. So when I did the exercise for sharing notebooks: The last step took me to the jupyter place and I had to myself click on the files there to see what they had in them. Is this what we wanted to happen?

    • Yes. One can let the binder link also link directly to the notebook instead of the overview. But this was the point of the exercise: being able to share a notebook with others who only need a browser -OK thanks!
      • See "Path to notebook file (optional)" at https://mybinder.org/.

  2. Please help answering Q55.

    • Looking ...

  3. Just one, maybe idiot, question. When we add the binder launcher on our repository, where our code is being run? So, in case I want to install R and Julia on the Jupyter, I can do the same way we did earlier, on the terminal of our computers? Or should we do on the terminal from the jupyter environment?

    • It is a good question! It is running "in the cloud" - more specifically: inside a Docker container that is created and later destroyed again.
    • If you want to share an R code via Binder: https://coderefinery.github.io/jupyter/sharing/#sharing-r-markdown-studio-projects
    • Julia example: https://github.com/binder-examples/demo-julia
    • In short: Binder will look for specific files and you need to add specific files to install dependencies Binder-side
      • Many thanks!
        • One nice thing about binderizing something (not sure it was mentioned) is that it forces me to find out what the dependencies even are because otherwise it won't run on Binder at all while it may still run on my computer ("runs on my machine"). It forces me to document dependencies which is a good thing to have in future.

Writing good readme files

https://coderefinery.github.io/documentation/writing-readme-files/

  1. Is it preferred to use commands in Git Bash or in the Jupyter lab terminal? Or just your preference?

    • Just preference. Both have times they are more convenient.

  2. If changes are made by a user in Binder, can these changes be viewed by others?

    • No, everything is lost in that case. Unless you save them, of course...

  3. I must have been distracted, sorry. But why did we jump directly to "writting good README files"?

    • We never have time for everything, we sometimes choose the most interesting things to discuss (and the rest is easier to read later).
    • So it's not that we are too slow (we/you are not!) but that we want to have more in the read-later material than we can and want to highlight in the 3.5 hours we have.

Exercise: Write a basic README until xx:31

https://coderefinery.github.io/documentation/writing-readme-files/#exercise-basic-readme

I am:

  1. How does GitHub recognises README.md and renders it? Is it a keyword?

    • Any .md etxnsion is rendered as Markdown. Same for other extensions.
      • What if I have multiple .md files? Will it all get rendered on GitHub? Sequentially?
        • Ah. In the repo default view, it looks for README.* (first? only?). But you can click on any file and that specific view does render that file according to extension.

  2. How can I see the other readme files created by other users?

    • some public projects:
    • numpy: https://github.com/numpy/numpy
    • jupyter: https://github.com/jupyter/jupyter
      • no I mean created within the exercise.
      • good idea! Let's see if anyone shares below...
    • If you want to share what you made, add a link here:
      • https://github.com/user/githubrepo.git

  3. Could you redirect me to the page that explains how to fork again?

    • on any github repo find on top right a button called "Fork" - is this what you meant?
    • In git-collaborative: https://coderefinery.github.io/git-collaborative/distributed/#exercise-part-1-creating-a-pull-request

  4. In general would you recommend to rename the GitHub branch main to master or rename the git branch master to main? To ensure compatibility between both.

    • Sort of up to you, what you want. But github + local git repo should be consistent...
      • Well, the push from my local git master branch to GitHub did not work. Because GitHub branch is called main, it made a new branch on GitHub called master. This is quite annoying and I want to be ahead of this next time.
      • Oh yeah. So, change one of them. Either locally (git branch -m main, you might want to check git set upstream branch) to make local, or change the default branch on Github.

  5. I forked the "coderefinery-githubrepo-example" but how do I then clone it to my computer?

    • You can git clone <repo-url> in a folder on your local system
    • Thanks! how do I then use git push to push my changes to the GitHub repo?
    • Make changes to the files then git add . then git commit -m "message" then git push
    • Thanks! I still cannot push my changes though. It says: fatal: Authentication failed for 'https://github.com/username/coderefinery-githubrepo-example'
    • make sure that the URL of the remote corresponds to you GitHub repository; to check the defined remotes: git remote -v.

  6. I added the readme.md file to my directory on GitHub but the markdown commands are not showing nicely i.e. ## and # and the lists. What's wrong?

    • would you like to share the repo link?
    • (github link)
      • thanks! don't worry we will redact addresses before publishing Q&A later today
      • you need a space after the "#": "# Rendering well", "#Not rendering well" -Thanks!
        • It's like in this document here. it is essentially the same "markup" language

Typealong Sphinx basics

https://coderefinery.github.io/documentation/sphinx/#exercise-sphinx-basics

  1. I missed this (was trying to fix a sphinx problem): seperate source and build directory or not?

    • We suggest to use the same directory this time, but both options work.

  2. How do I solve this?

    • error message:
PS C:\Users\user\AppData\Local\anaconda3\envs\coderefinery\doc-example> sphinx-build . _build
Running Sphinx v5.0.2

Extension error:
Could not import extension myst_parser (exception: No module 
named 'myst_parser')
PS C:\Users\user\AppData\Local\anaconda3\envs\coderefinery\doc-example>
  1. When I use sphinx-build . -build I get the error:
usage: sphinx-build [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
sphinx-build: error: the following arguments are required: outputdir, filenames

Did I miss a step? - You need the name of the build directory after "-build": sphinx-build . -build _build (if the build directory is called _build) - Oh nevermind, it is working now, thanks!

Exercise until xx:05

https://coderefinery.github.io/documentation/sphinx/#exercise-sphinx-basics

I am:

  1. Should the feature-a.md file go under the source directory?

    • No, at the same level as index.rst
    • (which might be the source directory?)

  2. I have earlier built sphinx doc with make html command. How is the sphinx-build _build different?

    • Basically the same, the make method automates some. If you look in Makefile you can see how.
    • We suggest sphinx-build in the exercise since it works on any OS without make installed.

  3. Question regarding jupyter notebooks: Could one attach the notebook plus output as appendix section? And if so how?

    • to the Sphinx project? Yes! The myst-nb extension does this: https://myst-nb.readthedocs.io/en/latest/index.html
    • I mean in an overleaf project.
    • Ah. That I'm less sure of, you probably need to export it in a way that LaTeX can read. Search around and see what you can find.
    • Also possibly interesting/relevant: https://jupyterbook.org/

  4. Can you elaborate on the difference between ReadTheDocs and Sphinx?

    • Sphinx is the thing that goes source → HTML
    • ReadTheDocs is online service that automates it: has users and projects, gets Github notifications and automatically rebuilds, provides the web servers. It runs sphinx (or other things) for you.
    • Follow-up: Does readthedocs support other tools for documentation creation?
      • Mkdocs I think. The "GitHub pages" method we will see next can do almost anything.
    • Follow-up 2: Does readthedocs have tools to generate docs on its own or deos it rely on those other "engines"?
      • Those other engines. (some customization. RTD releases sphinx_rtd_theme which is also usable without RTD, like we do.)
    • Thanks!

  5. Perhaps in the instruction the _build folder will be created after running the executable sphinx, right?

    • Yes.

  6. Unfortunately, I have created the feasure-b but it does not show in the webpage. Why is that, do you think?

    • Hm. Is it linked in the index? No.
    • Try adding to the toctree.
(coderefinery) C:\Users\user\AppData\Local\anaconda3\envs\coderefinery\doc-example>sphinx-build
usage: sphinx-build [OPTIONS] SOURCEDIR OUTPUTDIR [FILENAMES...]
sphinx-build: error: the following arguments are required: sourcedir, outputdir, filenames"

.. toctree::
  :maxdepth: 2
  :caption: Contents:

  feature-a.md
  1. Is there a way to show line numbers in nano? I ask since I received the following message:
sphinx-build . _build 
Running Sphinx v5.3.0

Configuration error:
There is a syntax error in your configuration file: invalid syntax (conf.py, line 19)

Deploying Sphinx to Github pages

https://coderefinery.github.io/documentation/gh_workflow/ (if this is too fast, you can also watch it as a demo and try later) (this is actually what CodeRefinery does for almost all of its websites!)

  1. kitty yay :heart_eyes_cat: : ooo

    • Pspsps

  2. Is it normal that the GitHub documentation updates with a delay? My first edit showed up there only when I pushed the second edit. Thanks, I was expecting it to show up immediately -- it's there now.

    • after push, click at "Actions" on top of repository page to see whether it is still building. this can take minutes (which is not a problem in real projects)
    • When building with GitHub pages, the whole GitHub action that is contained in the documentation.yaml is run. This is what takes some time, even for a small document.
    • Then it has to copy to web servers. Then distribute through the world. Then expire the old version.

  3. So GitHub pages does not build the docu but host static websites?

    • Yes. it will happily serve html built by whatever (there are so many different tools). but then the building can be done as part of GitHub Actions.
    • Thanks!

  4. Can you explain the deploy action in the yaml file and the use of the gh-pages branch?

    • Easier question first: gh-pages branch is standard branch name for github pages to serve from. historically it had to be called like that but these days you can configure to serve html from any branch. one thing that I invite you to verify I that it really created a branch called "gh-pages".
    • The Deploy part of the action reuses an action written by somebody else
      • Link for that building block: https://github.com/peaceiris/actions-gh-pages
      • We include it with: "uses: peaceiris/actions-gh-pages@v3"
      • And that action "building block" uses variables which we define under "with"
      • In short we could have written all ourselves but we reuse good building blocks written by others that can be reused in many similar projects
    • Did this answer the question?
      • Well, I need to read through that, including the actions. It seems strange to me, that using such a "headline" feature of GitHub requires so much custom code.
      • The upside of it is the flexibility to implement basically any action possible.

  5. How to remove the public documentation?

    • The GitHub pages website?
      • Yes.
        • One way is to remove the repository, then it's gone.
        • Less desctructive: go to settings and pages where you can disable it again.
          • Found the option "Unpublish...". Thanks!

Feedback, day 5

Tomorrow, we have one more interactive lesson on testing (using Git/Github Actions more + coderefinery Conda environment). Then one which is "sit back and watch." Together they are a great summary and worth watching by almost anyone!

Today was:

One good thing about today:

One thing to be improved next time:

Any other comments?:

Funding

CodeRefinery is a project within the Nordic e-Infrastructure Collaboration (NeIC). NeIC is an organisational unit under NordForsk.

Privacy

Privacy policy

Follow us

Contact

support@coderefinery.org

Improve this page

Source code