Dataframes from multimodal logs
Rerun 0.19 brings two huge new features; the ability to view and query back Rerun data as dataframes, and part one of supporting encoded video data.
The multimodal data stack
Open source log handling and visualization for spatial and embodied AI. Managed infrastructure to ingest, store, analyze, and stream data at scale with built-in visual debugging. Fast, flexible, and easy to use.
Rerun is open source
6,200
Start visualizing in seconds
Powerful and flexible visualization for spatial and embodied AI that's shockingly easy to get started with. Simple no sign-up installation and minimal code to get up and running.
pip install rerun-sdk
rerunInfrastructure to simplify your data engine
Run & record
Run your systems and record data for analysis and training. Use the Rerun SDK to log data or interpret existing log files.
Find issues
Use the Rerun viewer to understand behavior and pinpoint issues. Extract signals from logs for further analysis.
Improve & deploy
Visualize training and eval, debug prototypes, and extract time aligned training samples from messy logs.
How it works
1
Model
Use the SDK to model your data and write it to storage or a live viewer. Rerun's data model, a time aware Entity Component System, makes common scenarios simple but is flexible enough to handle custom data.
import rerun as rr
rr.init("my_data_generating_application")
rr.connect() # Connect to a remote viewer
…
rr.log("points", rr.Points3D(positions))
rr.log("camera", rr.Transform3D(pos, rot))
rr.log("camera/image", rr.Pinhole(intrinsics))
rr.log("camera/image", rr.Image(tensor))
rr.log("reprojection_error", rr.Scalar(err))2
View
Get powerful interactive visualizations of live and recorded data. Time travel through your data, with the industry's fastest multimodal viewer. It's open source and runs both natively and in the browser.
3
Build
Build layouts and customize visualizations directly through code or interactively in the UI. Build visualization into your tools by embedding the viewer in your Rust, JS, or Gradio apps.
4
Query
Use Rerun's query APIs to extract time-aligned datasets from messy robotics recordings. Rerun returns Apache Arrow data that plugs in to your favorite dataframe library for further analysis.
import rerun as rr
…
# Create an aligned dataframe based on time-points where an Image was logged
aligned_samples = rr.dataframe
.load_recording("raw.rrd")
.view(index="sensor_time", contents="/**")
.filter_is_not_null("/camera/image:ImageBuffer")
.fill_latest_at()
.select()
# Write aligned samples to a parquet file for later training
with pq.ParquetWriter("train.parquet", aligned_samples.schema()) as writer:
for chunk in aligned_samples:
writer.write_batch(chunk)
The community loves building on Rerun
LeRobot
LeRobot is Huggingface’s State-of-the-art AI for real-world robotics project. They are using Rerun as an integrated part of their visualization tools.
LeRobot is Huggingface’s State-of-the-art AI for real-world robotics project. They are using Rerun as an integrated part of their visualization tools.
Go to project →
Project Aria is a research platform developed by Meta Reality Labs Research to push the state of the art in egocentric AI research. Rerun is used to visualize sequences in their Aria Dataset Explorer.
Go to project →
Brush is a 3D reconstruction engine using Gaussian splatting developed by Arthur Brussee at Deepmind. It is written in Rust and aims to be highly portable, flexible and fast. Rerun is used for visualization during training.
Go to project →
The kornia-rs crate is a low level library for Computer Vision written in Rust 🦀 by the Kornia team. You can use the library to perform image I/O, Rerun vizualisation and other low level operations in your machine learning and data-science projects in a thread-safe and efficient way.
Go to project →
Open source vs commercial
Open source
Visualization and simple log handling
Dual licensed under MIT and Apache 2
Join on GitHub- The fastest & easiest to use multimodal visualizer out there
- Visualizer runs native, on the web and is embeddable in notebooks and web apps
- An SDK for modeling and handling multimodal logs
Commercial
Data management at scale
Under development with select design partners
Sign up for waitlist- Data platform for embodied AI where all data is instantly visualizable and debuggable
- Ingestion, storage engine and index management for large scale data
- Dataset management for both recordings and structured tables
Our blog
Exploiting column chunks for faster ingestion and lower memory use
To enable working with more kinds of datasets in Rerun, the 0.18 release introduces a new column oriented API and performance improvements to help handle larger time-series.
Defaults and overrides for any data
Rerun 0.17 introduces blueprint defaults and overrides, bringing a massive step up in user control. For any visualizer input, you can now use the blueprint to set default values or override the data you logged.