Video

This video from CS 533 @ Boise State University discusses the material here. It is not a replacement for this document, and talks primarily about Python, but is hopefully a useful supplement if you prefer video content (slides also available):

Standalone Documents

As I discuss in the video, a notebook is first a document — a document that incorporates code and visuals to present data with written explanation.

This means that you need to provide text that walks your reader through the story of your data analysis: what you are doing, why, how, and what we learn. This does not mean you should to describe every little detail (as too much detail can actually make it harder to read), but you need to provide the context for what the analyses mean and what we can learn from them.

In the context of a course assignment, the solution notebook should be readable even without having read the original assignment description: they should stand alone.

The document also needs to include the code outputs, where applicable — the reader should be able to read it without re-running the code, although the code is there so that they can. Without the output, the notebook does not work as a communicative document.

Complete Runs

The final, exported, submitted version of a notebook should contain the outputs from a complete run: make sure that re-running the notebook from top to bottom (e.g. “Restart and Run All” in Jupyter) works, and that the outputs included for submission are the results of this. This makes sure you submit the notebook in a working state, and that the results still match the code as written. Otherwise, we can get situations where e.g. you change a data file, but don’t rerun everything, and therefore some outputs do not reflect the data file loaded in the data load code.

Formatting Notebook Text

In the common notebook tools, the text content (text cells in Jupyter, document content in Quarto or Rmd) of a notebook is formatted with Markdown. LaTeX math markup, delimited with $ characters, is also widely supported. For example, this code:

The equation $a^n + b^n = c^n$ does not hold for integers $n>2$.

will yield:

The equation $a^n + b^n = c^n$ does not hold for integers $n>2$.

Notebooks also support block-mode math, delimited with $$ or \[.

Use Markdown formatting judiciously to highlight important points and make your notebooks easier to read. For example, using code formatting to mark up Python or R functions is often helpful:

The `train_test_split` function from SciKit Learn will helpfully
partition our data for us.

This renders as something like this, except using the notebook tool’s theme instead of my website’s:

The train_test_split function from SciKit Learn will helpfully partition our data for us.

Markdown also supports strong emphasis (bold) and emphasis (italics), which can be very useful for highlighting your argument. In addition to the common Markdown syntax, Jupyter and Quarto both support various extensions, such as strikethrough.

Markdown also supports **strong emphasis** (bold) and *emphasis* (italics),
which can be very useful for highlighting your argument.  In addition to the
common Markdown syntax, Jupyter and Quarto both support extensions, such as
~~strikethrough~~.

One thing that is important to pay attention to is use of section headings, as discussed in the video. Section headings are indicated with # characters, as in:

# Document Title

## Level 2 heading

Section headings are a crucial tool for structuring your document and making it easier to read. It’s important to note, however, that these have actual meaning: ## does not mean “large bold font”, it means “level 2 heading”. Properly structuring headings makes your document easier to read (see above, that a notebook is a document), and also enables tooling that to support navigating the document. JupyterLab and extensions to the notebook server both provide notebook outlines using the section headings, as do RStudio and Visual Studio Code. Section headings should also be short.

Finally, it is often helpful to use lists, either numbered or bulleted. For further reference on Markdown features and syntax, see:

• CommonMark syntax — the common Markdown syntax supported almost everywhere.
• GitHub Flavored Markdown (GFM) — superset of CommonMark mostly supported by Jupyter.
• Jupyter MarkDown docs — brief docs on using Markdown in Jupyter
• Pandoc manual — Pandoc is document processing tool for a heavily extended version of Markdown (along with other formats). Quarto notebooks are written in Pandoc, and Jupyter uses Pandoc to create LaTeX (and LaTeX-derived PDFs).

Rmd and Rnotebook mostly use CommonMark and GFM syntax.

Process

I recommend leaving time before something is due to go back through the notebook and clean it up for final presentation.

Sometimes it works best to start with the notebook you have and delete unnecessary code, remove excess debugging outputs, and improve the writing.

Sometimes it works best to start a fresh notebook, start putting together the structure, and copy over the code you actually need for the final solution.

Either way, you should produce a final notebook that is:

• readable
• executable from top to bottom
• clean of extraneous or unnecessary outputs, particularly without explanation

This last point is to avoid the “sea of charts” effect. If there are a lot of charts and tables that don’t advance your story, it is much harder to read. Not every output you created in the process of figuring out how to solve the problem will be useful to your reader. Additional debugging or deep-dive outputs can be moved to a separate file (that should also be executable!) and linked as an appendix to your main report.

Exporting Notebooks

Once you have your notebook ready and complete, you usually want to export it to a standalone file so that you can share it, submit it as an assignment solution, etc. without requiring the reader to open it in the notebook server (and for Quart, Rmd, or Rnotebook, an export is the only way to provide a file that includes the outputs, since they are not saved in the source notebook file).

Checklist

This checklist is to help you ensure your notebook is well-structured and well-written. I may expand or revise it as we progress through the semester.