Notebook: 2D neural fields

Overview overview

Rerun can be used directly within Jupyter notebooks. Many additional environments beyond Jupyter are supported such as Google Colab or VSCode.

You need the separate support package rerun-notebook to use this feature. Typically this is installed using:

pip install "rerun-sdk[notebook]"

Check out the minimal notebook example for a quick start.

Background background

In this notebook we are fitting a simple neural field to a 2D image. The neural field is a simple multilayer perceptron with optional positional input encoding. The image is sampled uniformly and the network is trained to predict the color given the pixel position. To visualize the progress of the training we log the loss and regularly densely query the network to retrieve the image encoded in the network weights.

Using the notebook we can interactively try different learning rates, losses, and network architectures to see how they affect the training process.

Running in Jupyter running-in-jupyter

The easiest way to try out the notebook is to use Jupyter.

First, install the requirements (this includes Jupyter, the Rerun SDK, and the notebook support package)

pip install -r requirements.txt

Then, open the notebook

jupyter notebook neural_field_2d.ipynb

Follow along in the browser that opens.