521 - Computing Platforms for Data Science¶

Master of Data Science 2021/22¶

University of British Columbia¶

Welcome to computer Platforms for Data Science! My name is Florencia D'Andrea and I will your instructor this term.

Welcome!

Florencia D'Andrea¶

E-Mail fdandrea@cs.ubc.ca
Slack channel #521_platforms-dsci
Office hours Fridays 10-11 am

The Turing Way Community, & Scriberia(2020)

In [1]:
   
![](https://media.giphy.com/media/l0HlOBZcl7sbV6LnO/giphy.gif?cid=790b7611c11622650af980c85872310bbf7505eaf6aa2f40&rid=giphy.gif&ct=g)

[Giphy](https://giphy.com/gifs/book-reading-library-l0HlOBZcl7sbV6LnO)

  File "C:\Users\FLOREN~1\AppData\Local\Temp/ipykernel_23784/1295321644.py", line 3
    [Giphy](https://giphy.com/gifs/book-reading-library-l0HlOBZcl7sbV6LnO)
                 ^
SyntaxError: invalid syntax

week 0¶

  1. Introduction to JupyterLab (You are here!)

Lab 0: Introduction to MDS homework and practice with Jupyter Notebooks

week 1¶

  1. Introduction to MDS software
  2. Getting groovy with Git and GitHub

Lab 1: Introduction to MDS homework and practice with the shell, Git, and GitHub

week 2¶

  1. Branches, forks, and streams... Welcome to the Git nature walk!
  2. Syncing with an upstream remote, time travelling with git reset, and pattern matching with regex

Lab 2: Branches and merge conflicts

week 3¶

  1. RStudio and R Markdown
  2. YAML, Templates, Presentations, and Notebook formats

Lab 3: Introduction to RStudio and R Markdown for reports and presentations

week 4¶

  1. SSH, filenames, and project organization
  2. Getting help, virtual environments, and some tips and tricks

Lab 4: RStudio projects and driving Git from RStudio

Learning Goals¶

  • Navigate their computer's filesystem via Bash.
  • Effectively use local and remote version control software (Git and GitHub).
  • Write regular expressions for pattern matching.
  • Create reproducible literate Python and R code documents (e.g., reports and presentations) using Jupyter and R notebooks.
  • Write Markdown syntax within literate code documents.

Learning Goals¶

  • Be comfortable using JupyterLab and RStudio.
  • Use virtual environments for R and Python.
  • Organize projects and name files in a sound manner.
  • Setup and authenticate via ssh keys
  • Create minimal reproducible code examples when asking for help.

Teaching Assistants¶

  • Yi Xuan Qi

  • Matthew Nguyen

  • Pouya Ahmadvand

  • Shan Lu

  • Abdul Muntakim Rafi

  • Yuxin Tian

  • Ngoc Bui

Support¶

  • Joel Ostblom

The Turing Way Community, & Scriberia(2020)

This year we have X teaching assistants that are going to help you in the labs. Names

Week 0¶

Assignments for Sep 8¶

  • MDS Policies Quiz
  • Software installation
  • Lab0

Lecture 0¶

Introduction to JupyterLab¶

After this lecture you will be able to:

  • Launch JupyterLab
  • Use the Notebook interface inside JupyterLab
  • Know your way around the JupyterLab user interface

What is JupyterLab?¶

The most rudimentary interaction with programming languages such as R and Python is via interactive shells run from a terminal.

The most rudimentary interaction with programming languages such as R and Python is via interactive shells run from a terminal. This provides access to the full functionality of the language, but is a barebones experience without any conveniences added. If you want to edit a script or view a plot that you created, you need to open a text editor and image viewer separately.

Integrated Development Environment (IDE)¶

Provide all the tools you need in one place

If you prefer a more holistic experience, with many of conveniences nicely organized in the same interface, you can use an integrated development environment (or IDE for short). IDEs often include a shell, a file browser, debugging tools, a text editor with autocompletion and syntax highlighting, and an area where plots show up. A core idea of IDEs is to provide all the tools you need in one place, and we will see later today how it is also possible to things like version control from within your IDE. Commons IDEs that you might have heard of include Visual Studio Code and RStudio, and today we will be learning about Jlab

After this lecture you will be able to:

  • Launch JupyterLab

  • Use the Notebook interface inside JupyterLab

  • Know your way around the JupyterLab user interface

JupyterLab, is and IDE to work with many programming languages, including R, Python, Julia, and many more. A core concept tightly linked to JupyterLab are Jupyter Notebooks, which will be one of the main topics in this talk, but first, let's see how we can launch JupyterLab. (DEMO)

How to access Jupyter Lab?¶

(1) UBC Server | (2) Installed version on your computer

How to access Jupyter Lab?¶

(1) UBC Server¶

  • Enter and login https://ubc.syzygy.ca/
  • Change the end of the URL manually from /tree to /lab

https://ubc.syzygy.ca/jupyter/user/fdandrea/lab

How to access Jupyter Lab?¶

(2) Installed version on your computer¶

Open the terminal and run

Jupyter lab

  • We could install and run JupyterLab just like any other program on Windows, MacOS, or Linux.

  • However, one of the advantages of JupyterLab is that it is easy to use without installing anything.

  • To try JupyterLab, use your web browser to visit the first Canvas assignment and login with your CWL (Firefox, and Chrome are supported)

    • Thanks to that JupyterLab is built entirely using web technologies, it can be rendered in a browser without the need to install anything.

  • Note that Jupyter Lab runs by default in a browser whether you run it locally or you are using it through UBC's servers. If you are running Jupyter Lab in your machine, the URL could appear as https://localhost:8888/lab. In this case, "localhost" it is not referring to a webpage, if not to your computer.

After this lecture you will be able to:

  • Launch JupyterLab

  • Use the Notebook interface inside JupyterLab

  • Know your way around the JupyterLab user interface

Using Jupyter Notebooks¶

The Notebook provides and interface where you can mix text, code, mathematical expressions, plots, videos, and more, all in the same file.

We're greeted by the launcher tab where we see that we can start either a Notebook or Console for Python or R, as well as some other utility programs. Let's start by explaining one of the most popular options, the Jupyter Notebook. The Notebook provides and interface where you can mix text, code, mathematical expressions, plot output, videos, and more, all in the same file. So instead of the traditional IDE experience where you would write code in a text file and then have figures pop up in a different panel, this information now all resides in the same document, which facilitates reproducibility and collaboration. The Notebooks can be exported to many formats, including PDF and HTML, which makes it easy to share your project with anyone.

Live Demo¶

The cell that is encircled in blue is where we can input Python code, click here and type any mathematical expression, and then run the cell by clicking the play button in the top toolbar: In [1]: 3 + 4 Out[1]: 7 As you can see, the output is returned just under input, and a new input cell was created. We could also have clicked the plus sign to create a new cell. Here, we can do anything we can do in Python, e.g. variable assignment: In [2]: a = 5 There is no output because we just assigned a value to a variable, without asking for the value of that variable, which we can do by typing out the variable name: In [3]: a Out[3]: 5 Jupyter Notebooks also supports editing code on multiple lines, so we could have done this instead: In [4]: a = 3 a Out[4]: 3

The Turing Way Community, & Scriberia (2020)

Working with text via Markdown cells¶

Heading level three¶

  • A bullet point
  • Emphasis in italics
  • Strong emphasis in bold
  • This is a link to learn more about markdown.
  • Support for $\LaTeX$ equations: $$f'(a) = \lim_{x \to a} \frac{f(x) - f(a)}{x-a}$$

You can change the input cell type from Python code to Markdown by clicking on the little dropdown menu in the toolbar that reads "Code".

Markdown is a simple formatting system which allows you to document your analysis within the notebook. This is also great for creating tutorials and even books as we will see later. It is a plain text format that includes brief markup tags that indicates how text should be rendered, e.g. * indicates italics and ** indicates bold typeface. If you have commented on online forums or used a chat application, you might already be familiar with markdown. Below is a short example of the syntax:

Static figure creation¶

In [ ]:
# Please note, you don't have to understand what is going on in this cell
# it is just to show you how a static plot looks inside JupyterLab

import numpy as np
import matplotlib.pyplot as plt

N = 50
x = np.random.rand(N)
y = np.random.rand(N)
colors = np.random.rand(N)
area = (30 * np.random.rand(N))**2  # 0 to 15 point radii

plt.scatter(x, y, s=area, c=colors, alpha=0.5)

To show an example of how plots are rendered, we will use the seaborn plotting package which is a high level interface to the more widely known matplotlib package. This is just to illustrate how plots show up in the notebook rather than a tutorial on how to plot in Python so I will not go into details on what these commands mean.

The above is a static figure, which means an image is created and included in the notebook. To save this image, you can either use a save command in the plotting library or you can hold shift and right click. Just right clicking is the JupyterLab menu, but holding shift brings up the browser menu, which includes the common options for images such as open in new tab and save.

Exporting notebooks¶

The combination of code, plots, and markdown is powerful. With these three elements, you can keep all your analysis code and interpretations in the same document, instead of spread across different files. Notebooks are excellent tools to build entire reports, since they can contain formatting such as a table of contents, links to sections and files, footnotes, images, bibliographies, and even videos, which we will learn more about later in the workshop. You can export the notebook to different file types via File --> Export Notebook As.... (When you export to PDF for the first time on your own machine, you might be asked to download the Latex packages, which are needed to create the PDF.) If we would export this notebook the HTML, we would still retain the interactivity of the plots we created!

After this lecture you will be able to:

  • Launch JupyterLab

  • Use the Notebook interface inside JupyterLab

  • Know your way around the JupyterLab user interface

The JupyterLab sidebar¶

  • Now that you know the basics of how to work inside Jupyter Notebooks, let's continue explorer the JupyterLab user interface.
  • The puzzle piece at the bottom is where you can install extensions.
  • The next icon just shows the tabs, same as what you see on top.
  • The next icon is a bit more advanced as it allows you to add metadata to cells.
    • One aspect worth highlighting here is the ability to denote cells as part of a slide show, that you can download via File -> Export notebook as.
  • The commands section is really helpful if you can't remember a shortcut or where your command is in the menu.
    • It also shows the keyboard shortcuts for each command.
  • The stop sign icon shows an overview of all running applications.
  • The topmost icon in the sidebar to the left shows the file tree of your current folder, highlights include:
    • the icon for uploading files from your computer to the UBC Server (or your folder if you run locally)
    • right clicking a file to download it to your computer
    • clicking the + sign takes us back to the launcher menu where we started

Additional JupyterLab applications¶

jupyterLab can run applications other than notebooks, e.g. there is a Python console, a text editor, and a terminal emulator. These can be opened via the launcher page or File --> New. Applications can be placed side by side by dragging and dropping their windows, so we could be running a terminal and notebook next to each other.

How to get help in the notebook¶

gif extracted from Language Server Protocol integration for Jupyter(Lab)

gif extracted from Language Server Protocol integration for Jupyter(Lab)

One application that is especially helpful to run next to a notebook is the Contextual Help. This application displays documentation automatically as you type. When you're using unfamiliar packages and functions, it is a good habit to leave the Contextual Help open next to the notebook. If you don't like having a split screen, you can instead press Shift + Tab to bring up a help dialogue. JupyterLab also supports tab completion, you can start typing a name and then press tab to see suggestions to expand to. Additional help is available via the "Help" menu, which links to useful online resources (for example Help --> JupyterLab Reference).

If you need to restart the server, you can go to File -> Hub control panel -> Stop my server and then click Start my server -> Launch server.

All your files will be saved between restarts, but any Python packages you have installed yourself will be reset, so you need to contact UBC IT to have additional Python packages installed with persistence.

Thank you!¶

Slides: Jupyter Notebook + GitHub Pages!¶

Images¶

The Turing Way Community, & Scriberia. (2020). Illustrations from the Turing Way book dashes. Zenodo. https://doi.org/10.5281/zenodo.3695300