Jupyter Notebooks

Introduction

Jupyter notebook, formerly known as the IPython notebook, is a flexible tool that helps you create readable analyses, as you can keep code, images, comments, formulae and plots together.

Jupyter is quite extensible, supports many programming languages.

Jupyter Icon

Project Jupyter was born out of the IPython project as the project evolved to become a notebook that could support multiple languages - hence its historical name as the IPython notebook. The name Jupyter is an indirect acronym of the three core languages it was designed for: JUlia, PYThon, and R and is inspired by the planet Jupiter.

When working with Python in Jupyter, the IPython kernel is used, which gives us some handy access to IPython features from within our Jupyter notebooks.

We're going to show you some of the tips and tricks to make your life easier working with Jupyter.

Getting Started

Once if you are inside the workspace click on the Jupyter Notebooks bookmark to get to the jupyter Notebooks. You will see some sample notebooks that come pre-loaded in your workspace.

The following Gif will show you all the tabs within the Jupyter notebooks environment.

Jupyter Notebooks Tabs

The "Files" tab is where all your files are kept, the "Running" tab keeps track of all your processes and the third tab, "Clusters", is provided by IPython parallel, IPython's parallel computing framework. It allows you to control many individual engines, which are an extended version of the IPython kernel.

You probably want to start by making a new notebook.

You can easily do this by clicking on the "New button" in the "Files tab". You see that you have the option to make a regular text file, a folder, a terminal and a notebook with a terminal of your choice.

Let's start first with the regular text file. When it open up, you see that this looks like any other text editor. You can toggle the line numbers or/and the header, you can indicate the programming language you're writing and you can do a find and replace. Furthermore, you can save, rename or download the file or make a new file.

Jupyter Editor

You can also make folders to keep all your documents organized together. Just press the "Folder" option that appears when you press the "New" button in your initial menu and a new folder will appear in your overview. You can best rename the folder instantly, as it will appear as a folder with name 'Untitled Folder'.

Thirdly, the terminal is there to support browser-based interactive terminal sessions. It basically works just like your terminal or cmd application! Give in python into the terminal, press ENTER, and you're good to go.

If you want to start on your notebook, go back to the main menu and click on the kernel that you're interested in the "Notebook" category.

You will immediately see the notebook name, a menu bar, a toolbar and an empty code cell:

Jupyter Terminal

You can immediately start with importing the necessary libraries for your code.

After, you can add, remove or edit the cells according to your needs. And don't forget to insert explanatory text or titles and subtitles to clarify your code! That's what makes a notebook a notebook in the end.

Jupyter Add Edit Cells

Tips and Tricks

  1. Keyboard Shortcuts

    As any power user knows, keyboard shortcuts will save you lots of time. Jupyter stores a list of keybord shortcuts under the menu at the top: Help > Keyboard Shortcuts, or by pressing H in command mode (more on that later). It’s worth checking this each time you update Jupyter, as more shortcuts are added all the time.

    Keyboard Shortcuts

    Another way to access keyboard shortcuts, and a handy way to learn them is to use the command palette: Ctrl + Shift + p inside your workpsace. This dialog box helps you run any command by name - useful if you don’t know the keyboard shortcut for an action or if what you want to do does not have a keyboard shortcut.

    Some of the favorites

    • Esc will take you into command mode where you can navigate around your notebook with arrow keys.
    • While in command mode
      • A to insert a new cell above the current cell, B to insert a new cell below.
      • M to change the current cell to Markdown, Y to change it back to code
      • D + D (press the key twice) to delete the current cell
      • Enter will take you from command mode back into edit mode for the given cell.
      • Tab will show you the Docstring (documentation) for the the object you have just typed in a code cell - you can keep pressing this shortcut to cycle through a few modes of documentation.

        Keyboard Shortcut Tab
      • Ctrl + Shift + - will split the current cell into two from where your cursor is.
      • Esc + F Find and replace on your code but not the outputs.
      • Esc + O Toggle cell output.
      • Select Multiple Cells
        • Shift + J or Shift + Down selects the next sell in a downwards direction. You can also select sells in an upwards direction by using Shift + K or Shift + Up
        • Once cells are selected, you can then delete / copy / cut / paste / run them as a batch. This is helpful when you need to move parts of a notebook.
        • You can also use Shift + M to merge multiple cells.

          Select Multiple Cells
  2. Pretty Display of Variables

    The first part of this is pretty widely known. By finishing a Jupyter cell with the name of a variable or unassigned output of a statement, Jupyter will display that variable without the need for a print statement. This is especially useful when dealing with Pandas DataFrames, as the output is neatly formatted into a table.

    What is known less, is that you can alter a modify the ast_note_interactivity kernel option to make jupyter do this for any variable or statement on it’s own line, so you can see the value of multiple statements at once.

    Pretty Display Variables
  3. Plotting in notebooks

    There are many options for generating plots in your notebooks.

    • matplotlib (the de-facto standard), activated with %matplotlib inline - Here’s a Dataquest Matplotlib Tutorial.
    • %matplotlib notebook provides interactivity but can be a little slow, since rendering is done server-side.
    • Seaborn is built over Matplotlib and makes building more attractive plots easier. Just by importing Seaborn, your matplotlib plots are made ‘prettier’ without any code modification.
    • mpld3 provides alternative renderer (using d3) for matplotlib code. Quite nice, though incomplete.
    • bokeh is a better option for building interactive plots.
    • plot.ly can generate nice plots - this used to be a paid service only but was recently open sourced.
    • Altair is a relatively new declarative visualization library for Python. It’s easy to use and makes great looking plots, however the ability to customize those plots is not nearly as powerful as in Matplotlib.
    Bokeh Plot Diagram
  4. IPython Magic Commands

    The %matplotlib inline you saw above was an example of a IPython Magic command. Being based on the IPython kernel, Jupyter has access to all the Magics from the IPython kernel, and they can make your life a lot easier!

    Magic Commands
    IPython Magic - %env: Set Environment Variables

    You can manage environment variables of your notebook without restarting the jupyter server process. Some libraries (like theano) use environment variables to control behavior, %env is the most convenient way.

    Magic env Command
    IPython Magic - %run: Execute python code

    %run can execute python code from .py files - this is well-documented behavior. Lesser known is the fact that it can also execute other jupyter notebooks, which can quite useful.

    Note that using %run is not the same as importing a python module.

    Magic run Command
    IPython Magic - %load: Insert the code from an external script

    This will replace the contents of the cell with an external script. You can either use a file on your computer as a source, or alternatively a URL.

    Magic Load Command
    IPython Magic - %store: Pass variables between notebooks.

    The %store command lets you pass variables between two different notebooks.

    Magic Store Command
    IPython Magic - %who: List all variables of global scope.

    The %who command without any arguments will list all variables that existing in the global scope. Passing a parameter like str will list only variables of that type.

    Magic who Command
    IPython Magic - Timing

    There are two IPython Magic commands that are useful for timing - %%time and %timeit. These are especially handy when you have some slow code and you’re trying to indentify where the issue is.

    %%time will give you information about a single run of the code in your cell.

    Magic timing Command
    IPython Magic - %%writefile and %pycat: Export the contents of a cell/Show the contents of an external script

    Using the %%writefile magic saves the contents of that cell to an external file. %pycat does the opposite, and shows you (in a popup) the syntax highlighted contents of an external file.

    Magic %%writefile Command
    IPython Magic - %prun: Show how much time your program spent in each function.

    Using %prun statement_name will give you an ordered table showing you the number of times each internal function was called within the statement, the time each call took as well as the cumulative time of all runs of the function.

    Magic prun Command
    IPython Magic - Debugging with %pdb

    Jupyter has own interface for The Python Debugger (pdb). This makes it possible to go inside the function and investigate what happens there.

    You can view a list of accepted commands for pdb here.

    Magic pdb Command
  5. Suppress the output of a final function

    Sometimes it’s handy to suppress the output of the function on a final line, for instance when plotting. To do this, you just add a semicolon at the end.

    Suppress Output
  6. Executing Shell Commands

    It’s easy to execute a shell command from inside your notebook. You can run any of the shell command by adding '!' in front of it.

    Shell Commands
  7. Using LaTeX for forumlas

    When you write LaTeX in a Markdown cell, it will be rendered as a formula using MathJax.

    Latex Formula

    Markdown is an important part of notebooks, so don’t forget to use its expressiveness!

  8. Run code from a different kernel in a notebook

    If you want to, you can combine code from multiple kernels into one notebook.

    Just use IPython Magics with the name of your kernel at the start of each cell that you want to use that Kernel for:

    • %%bash
    • %%HTML
    • %%python2
    • %%python3
    • %%ruby
    • %%perl
    Using a different kernel example
  9. Running R and Python in the same notebook

    The best solution to this is to install rpy2 (requires a working version of R as well), which can be easily done with pip:

    pip install rpy2

    You can then use the two languages together, and even pass variables inbetween:

    Multi Language Support
  10. Multicursor support

    Jupyter supports mutiple cursors, similar to Sublime Text. Simply click and drag your mouse while holding down Alt.

    Multi Cursor Support
  11. Jupyter-contrib extensions

    Jupyter-contrib extensions is a family of extensions which give Jupyter a lot more functionality, including e.g. jupyter spell-checker and code-formatter.

    Jupyter Contrib Extensions
  12. Sharing notebooks

    The easiest way to share your notebook is simply using the notebook file (.ipynb), but for those who don’t use Jupyter, you have a few options:

    • Convert notebooks to html file using the File > Download as > HTML Menu option.
    • Share your notebook file on PMP platform.
  13. Interactive Notebooks As Dashboards: Widgets

    The magic commands already do a lot to make your workflow with notebooks agreeable, but you can also take additional steps to make your notebook an interactive place for others by adding widgets to it!

    To add widgets to your notebook, you need to import widgets from ipywidgets:

    from ipywidgets import widgets

    That's enough to get started! You might want to think now of what type of widget you want to add. The basic types of widgets are text input, buttons, and input-based widgets.

    See an example of a text input widget below:

    Multi Cursor Support