Embed Rerun in notebooks

Rerun can embed the Rerun Viewer directly within IPython-style notebooks. This makes it easy to iterate on API calls as well as to share data with others.

Rerun has been tested with:

To begin, install the rerun-sdk package with the notebook extra:

pip install rerun-sdk[notebook]

This installs both rerun-sdk and rerun-notebook.

The APIs

When using the Rerun logging APIs, by default, the logged messages are buffered in-memory until you send them to a sink such as via rr.connect_grpc() or rr.save().

When using Rerun in a notebook, rather than using the other sinks, you have the option to use rr.notebook_show(). This method embeds the web viewer using the IPython display mechanism in the cell output, and sends the current recording data to it.

Once the viewer is open, any subsequent rr.log() calls will send their data directly to the viewer, without any intermediate buffering.

For example:

import rerun as rr
from numpy.random import default_rng

rr.init("rerun_example_notebook")

rng = default_rng(12345)

positions = rng.uniform(-5, 5, size=[10, 3])
colors = rng.uniform(0, 255, size=[10, 3])
radii = rng.uniform(0, 1, size=[10])

rr.log("random", rr.Points3D(positions, colors=colors, radii=radii))

rr.notebook_show()

This is similar to calling rr.connect_grpc() or rr.serve_grpc() in that it configures the Rerun SDK to send data to a viewer instance.

Note that the call to rr.notebook_show() drains the recording of its data. This means that any subsequent calls to rr.notebook_show() will not result in the same data being displayed, because it has already been removed from the recording. Support for this is tracked in #6612.

If you wish to start a new recording, you can call rr.init() again.

The notebook_show() method also takes optional arguments for specifying the width and height of the viewer. For example:

rr.notebook_show(width=400, height=400)

Reacting to events in the Viewer

It is possible to register a callback to be triggered when certain Viewer events happen.

For example, here is how you can track which entities are currently selected in the Viewer:

from rerun.notebook import Viewer, ViewerEvent

selected_entities = []


def on_event(event: ViewerEvent):
    global selected_entities
    selected_entities = []  # clear the list

    if event.type == "selection_change":
        for item in event.items:
            if item.type == "entity":
                selected_entities.append(item.entity_path)


viewer = Viewer()
viewer.on_event(on_event)

display(viewer)

Whenever an entity is selected in the Viewer, selected_entities is updated. The payload includes other useful information, such as the position of the selection within a 2D or 3D view.

For a more complete example, see callbacks.ipynb.

Working with blueprints

Blueprints can also be used with notebook_show() by providing a blueprint parameter.

For example

blueprint = rrb.Blueprint(
    rrb.Horizontal(rrb.Spatial3DView(origin="/world"), rrb.Spatial2DView(origin="/world/camera"), column_shares=[2, 1]),
)

rr.notebook_show(blueprint=blueprint)

Because blueprint types implement _ipython_display_, you can also just end any cell with a blueprint object, and it will call notebook_show() behind the scenes.

import numpy as np
import rerun as rr
import rerun.blueprint as rrb

rr.init("rerun_example_image")
rng = np.random.default_rng(12345)

image1 = rng.uniform(0, 255, size=[24, 64, 3])
image2 = rng.uniform(0, 255, size=[24, 64, 1])

rr.log("image1", rr.Image(image1))
rr.log("image2", rr.Image(image2))

rrb.Vertical(rrb.Spatial2DView(origin="/image1"), rrb.Spatial2DView(origin="/image2"))

Streaming data

The notebook integration supports streaming data to the viewer during cell execution.

You can call rr.notebook_show() at any point after calling rr.init(), and any rr.log() calls will be sent to the viewer in real-time.

import math
from time import sleep

import numpy as np
import rerun as rr
from rerun.utilities import build_color_grid

rr.init("rerun_example_notebook")
rr.notebook_show()

STEPS = 100
twists = math.pi * np.sin(np.linspace(0, math.tau, STEPS)) / 4
for t in range(STEPS):
    sleep(0.05)  # delay to simulate a long-running computation
    rr.set_time("step", sequence=t)
    cube = build_color_grid(10, 10, 10, twist=twists[t])
    rr.log("cube", rr.Points3D(cube.positions, colors=cube.colors, radii=0.5))

Some working examples

To experiment with notebooks yourself, there are a few options.

Running locally

The GitHub repo includes a notebook example.

If you have a local checkout of Rerun, you can:

$ cd examples/notebook/notebook
$ pip install -r requirements.txt
$ jupyter notebook cube.ipynb

This will open a browser window showing the notebook where you can follow along.

Running in Google Colab

You can open a copy of this notebook in Google Colab.

The first cell installs the latest Rerun release into the Colab environment and then restarts the runtime. Run the cells again after the restart.

Running in marimo

The same example is available as a marimo notebook:

$ cd examples/notebook/notebook
$ pip install "rerun-sdk[notebook]" marimo
$ marimo edit cube_marimo.py

marimo handles messages from the viewer only between cells, so a viewer cannot receive data until the cell that created it has finished. To stream data live, create the viewer in one cell and log to it from a later one. block_until_ready has no effect in marimo.

Troubleshooting: the viewer is blank or does not load

The notebook viewer is a widget that your browser downloads as two files: widget.js and the Viewer compiled to WebAssembly. By default, the browser downloads them from app.rerun.io, for the installed version of rerun-notebook. If the browser cannot reach that address (for example behind a firewall, or offline), the viewer stays blank.

Set the RERUN_NOTEBOOK_ASSET environment variable to choose where the files come from. Set it before you import rerun, since it is only read at import time (restart the kernel if you already imported it):

import os

os.environ["RERUN_NOTEBOOK_ASSET"] = "inline"

import rerun as rr

Value	Where the browser gets the widget
(not set)	From `app.rerun.io`.
`inline`	Sent by the kernel through the notebook connection. Works everywhere, but is slower to load and uses more memory.
`serve-local`	From a small HTTP server that the kernel starts on `localhost`. Only works when the browser and the kernel run on the same machine.
`https://example.com/…/widget.js`	From your own server, which must also serve `re_viewer_bg.wasm` next to `widget.js`.

When the kernel runs on another machine than your browser (a remote Jupyter server, VS Code Remote, or a hosted notebook service such as Databricks), serve-local does not work, because localhost in your browser is not the kernel's machine. Use the default if the browser can reach app.rerun.io, and inline if it cannot.

To host the files yourself, python -m rerun_notebook serve starts a server for them.

Limitations

Browsers have limitations in the amount of memory usable by a single tab. If you are working with large datasets, you may run into browser tab crashes due to out-of-memory errors. If you encounter the issue, you can try to use the save() API to save the data to a file and share it as a standalone asset.

Future work

We are actively working on improving the notebook experience and welcome any feedback or suggestions.