Jupyter magic

Jupyter Notebooks is a great interactive tool for creating fast and representative Data Science scripts. The Notebooks can not only be used for easy reporting and testing of algorithms, but also used for technical training as you can add various type of cells: code, markdown and raw. Today I want to write a bit about the magic that can further enhance your notebook and add usability.

Showing plots in the notebook

One of the advantages in Jupyter is the ease of showing plots. To enable doing that, you only need to call the %matplotlib inline magic. This is probably the most commonly used magic by Jupyter users. Let’s start with some basic code that will display a line plot with the magic included:

%matplotlib inline
import matplotlib.pyplot as plt

x = [0, 1, 2, 3, 4, 5]
y = [5, 4, 3, 2, 1, 0]

plt.plot(x, y)

And this is how the output looks in a Jupyter Notebook:

Passing data between notebooks

I like to keep things simple. If a notebook is starting to get too big, I find it usefull to throw eg. the data preparation in an additional notebook. But by default, the notebooks do not share the data, even if they run at the same time. This is where the %store magic comes in handy. Let’s say we read the data in Notebook A. This is the code we will run:

# notebook_a.ipynb
import pandas as pd

df = pd.read_csv('banana_data.csv')
%store df

Notice the magic at the end of the code. We need to pass the variable we are interested in storing. Next we can read the data in Notebook B as seen below:

# notebook_b.ipynb
%store -r df

Benchmarking code

Sometimes you want to compare two algorithms and see how fast both produce results. You might know the timeit function from Python already, but how to use that in Jupyter Notebooks? Let’s find out!

result = [x for x in range(0,10000)]

Will give an output structured like this:

407 µs ± 8.49 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Check out all the variables

At one point, we might get lost what is actually happening in the notebook and aren’t sure what the variables we coded are. We can investigate that using the %who_ls magic as an easy solution. Let’s say that we have the following variables in our notebook:

number = 123
text = 'I like bananas.'
other_text = 'I really do.'

If we run the %who_ls magic we will get the following result:


['other_text', 'text', 'number']

When we need to find variables of a specific type, we can still do that!

%who_ls str

['other_text', 'text']

Running external scripts

Jupyter magics also allow you to run external files in the Notebook. This can be useful, if you have a script that you want to test without copying it into the Notebook. To run code from a file, we need to use the %run magic. Let’s say we have a file named banana.py:

# banana.py
print('This is my banana script.')

The only thing we need to do to execute the code in our notebook is enter the following line:

%run banana.py


Of course the magics I have presented aren’t the only ones that are available. If you are interested in the subject, you can go to the official documentation on IPython magics. Alternatively you can use a question mark at the end of a magic, to see help, eg. %matplotlib? or %timeit?. What magics do you use? Feel free to give your examples in the comments!

Tell me what you think!