# Rerun
> Rerun is the Unified Data Layer for Physical AI: a platform for logging, visualizing, querying, transforming, and training on multi-rate, multimodal, time-series data in robotics, spatial computing, and embodied AI.
- Rerun has two parts: the open source **Rerun SDK** (libraries and tools for logging, storing, querying, visualizing, transforming, and training on multi-rate, multimodal data) and **Rerun Hub**, a commercial data catalog and storage engine that extends the SDK to large-scale datasets backed by object storage. Together they form the Unified Data Layer for Physical AI.
- **Rerun SDK is open source.** You can find the project on [GitHub](https://github.com/rerun-io/rerun). It is permissively licensed under MIT, so you can use it for personal and commercial projects.
- The Rerun SDK includes logging APIs for [Python](https://ref.rerun.io/docs/python), [C++](https://ref.rerun.io/docs/cpp), and [Rust](https://docs.rs/rerun/), the Rerun viewer, a CLI, catalog APIs, chunk processing APIs, and a [PyTorch dataloader](https://rerun.io/blog/data-layer-for-robot-learning).
- The Rerun viewer can either be installed and run natively, or in the browser.
- You can [try Rerun in your browser](/viewer).
## Docs
# Reference
The reference docs detail how to use the logging APIs and the viewer.
- [Types](https://rerun.io/docs/reference/types.md) - archetypes, components, and datatypes
- [Viewer](https://rerun.io/docs/reference/viewer/overview.md) - the Rerun Viewer UI
- [CLI manual](https://rerun.io/docs/reference/cli.md) - command-line interface reference
- [Python APIs](https://ref.rerun.io/docs/python) - Python SDK reference
- [Rust APIs](https://docs.rs/rerun/) - Rust SDK reference
- [C++ APIs](https://ref.rerun.io/docs/cpp) - C++ SDK reference
- [Web Viewer API](https://ref.rerun.io/docs/js/) - JavaScript/TypeScript web viewer API
- [Migration](https://rerun.io/docs/reference/migration.md) - guides for upgrading between versions
# Getting Started
Rerun helps robotics and Physical AI teams iterate faster: log from any sensor, visualize in the Viewer, query with dataframes, and train with a dataloader tailored to robotic learning β across one recording or many.
## Installation
`pip install rerun-sdk[dataplatform, dataloader]` bundles the **SDK** (log/query from code) and the **Viewer** (visualizer app). The optional dependencies support queries and training below.
For Rust, C++, see [Install Rerun](https://rerun.io/docs/getting-started/install-rerun.md) and [Set up a project](https://rerun.io/docs/getting-started/project-setup.md).
## Open the Viewer
`rerun` launches the Viewer.
Pass a file to open it directly:
```bash
rerun path/to/recording.rrd
```
Supports `.rrd`, `.mcap`, and [more](https://rerun.io/docs/getting-started/data-in/open-any-file.md).
Also available in-browser at [rerun.io/viewer](https://rerun.io/viewer).
## Scale across many recordings
Rerun's catalog organizes recordings as queryable [**segments**](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md).
The workflow: log (or convert) data to an `.rrd`, start a catalog server (or connect to an existing one if using the commercial Rerun Hub), register the `.rrd` as a segment, then visualize and query across recordings.
### Log
Save data to an `.rrd` see [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) for more details.
If you already have data in another format see our [how-to](https://rerun.io/docs/howto/logging-and-ingestion) for various examples converting to `.rrd`.
```python
import math
import rerun as rr
with rr.RecordingStream(
"rerun_example_getting_started", recording_id="run-1", send_properties=False
) as rec:
rec.save("run-1.rrd")
for t in range(10):
rec.set_time("step", sequence=t)
rec.log("/arm/shoulder", rr.Scalars(math.sin(t * 0.5)))
rec.log("/arm/elbow", rr.Scalars(math.cos(t * 0.5)))
```
### Start a catalog server
`rerun server` starts a local catalog on port `51234` (use Rerun Hub for persistent, multi-user storage), then connect from your code:
```bash
rerun server
```
```python
# `server_url` is the catalog URL β defaults to "rerun+http://127.0.0.1:51234"
# when running `rerun server` locally.
client = rr.catalog.CatalogClient(server_url)
```
### Ingest
Register an `.rrd` with a dataset so it shows up as a queryable segment.
```python
dataset = client.create_dataset("demo", exist_ok=True)
dataset.register([Path("run-1.rrd").absolute().as_uri()]).wait()
```
### Visualize
Point the Viewer at your server to browse every recording in the catalog.
See [Configure the Viewer](https://rerun.io/docs/getting-started/configure-the-viewer.md).
```bash
rerun rerun+http://127.0.0.1:51234
```
### Query
Query the catalog into a [DataFusion](https://datafusion.apache.org/) DataFrame. See [Query and Transform](https://rerun.io/docs/getting-started/data-out.md).
```python
df = dataset.filter_contents(["/arm/**"]).reader(index="step")
print(
df.select(
"rerun_segment_id",
"/arm/shoulder:Scalars:scalars",
"/arm/elbow:Scalars:scalars",
)
)
```
### Train
Connect a Dataloader to the server to generate training batches. See [Train](https://rerun.io/docs/getting-started/train.md).
```python
from torch.utils.data import DataLoader
from rerun.experimental.dataloader import (
DataSource,
Field,
NumericDecoder,
RerunIterableDataset,
)
ds = RerunIterableDataset(
source=DataSource(dataset=dataset),
index="step",
fields={
"shoulder": Field(
"/arm/shoulder:Scalars:scalars", decode=NumericDecoder()
),
"elbow": Field("/arm/elbow:Scalars:scalars", decode=NumericDecoder()),
},
)
for batch in DataLoader(ds, batch_size=4):
print(batch)
```
## If you're stuck
- Check the [troubleshooting guide](https://rerun.io/docs/getting-started/install-rerun/troubleshooting.md).
- [Open an issue](https://github.com/rerun-io/rerun/issues/new/choose).
- [Join the Discord server](https://discord.gg/PXtCgFBSmH).
# Developing Rerun
If you want to contribute and help develop Rerun, that work happens in the [Rerun repository](https://github.com/rerun-io/rerun). Also, make sure to join the [Rerun Discord](https://discord.gg/PXtCgFBSmH).
# Concepts
For a deeper understanding of how your logging data can be organized and visualized, it is useful to learn about the underlying concepts in Rerun.
- [How Does Rerun Work](https://rerun.io/docs/concepts/how-does-rerun-work.md) - understand the high-level architecture
- [Logging and ingestion](https://rerun.io/docs/concepts/logging-and-ingestion.md) - how data is structured and sent to Rerun
- [Visualization](https://rerun.io/docs/concepts/visualization.md) - how data is displayed in the Viewer
- [Query and transform](https://rerun.io/docs/concepts/query-and-transform.md) - how to query and transform data
- [Lenses](https://rerun.io/docs/concepts/query-and-transform/lenses.md) - extract, reshape, and reroute component data
- [Train](https://rerun.io/docs/concepts/train.md) - how to use Rerun data for training
# How-to
Guides for using Rerun in more advanced ways.
- [Logging and ingestion](https://rerun.io/docs/howto/logging-and-ingestion.md) - sending data to Rerun
- [Visualization](https://rerun.io/docs/howto/visualization.md) - displaying data in the Viewer
- [Query and transform](https://rerun.io/docs/howto/query-and-transform.md) - querying and transforming data
- [Train](https://rerun.io/docs/howto/train.md) - using Rerun data for training
- [Integrations](https://rerun.io/docs/howto/integrations.md) - integrating Rerun with other tools
# Log and Ingest
How-to guides for sending data to Rerun.
# Integrations
How-to guides for integrating Rerun with other tools.
# Extend Rerun
There are currently two major ways of extending Rerun. You can use Rerun with [your own custom data](https://rerun.io/docs/howto/logging-and-ingestion/custom-data.md), or [extend the Rerun Viewer](https://rerun.io/docs/howto/visualization/extend-ui.md) (currently Rust only).
The goal is for Rerun to become easy to extend at every level. For example, with plugins for
- data sources
- data stores
- data transforms
- custom rendering
- custom UI interactions
If you're in need of a particular kind of extension mechanism that we don't yet support. Head over to [GitHub](https://github.com/rerun-io/rerun/issues) and either upvote an existing issue or open a new one.
# Query and Transform
How-to guides for querying and transforming data in Rerun.
# Visualize
How-to guides for visualizing data in the Rerun Viewer.
# Building blueprints programmatically
For maximum control and automation, you can define [Blueprints](https://rerun.io/docs/concepts/visualization/blueprints.md) in code using the Python Blueprint API. This is ideal for:
- Creating layouts dynamically based on your data
- Ensuring consistent views for specific debugging scenarios
- Generating complex layouts that would be tedious to build manually
- Sending different blueprints based on runtime conditions
### Getting started example
This walkthrough demonstrates the Blueprint API using stock market data. We'll start simple and progressively build more complex layouts.
#### Setup
First, create a virtual environment and install dependencies:
**Linux/Mac:**
```bash
python -m venv venv
source venv/bin/activate
pip install rerun-sdk humanize yfinance
```
**Windows:**
```bash
python -m venv venv
.\venv\Scripts\activate
pip install rerun-sdk humanize yfinance
```
#### Basic script
Create `stocks.py` with the necessary imports:
```python
#!/usr/bin/env python3
import datetime as dt
import humanize
import pytz
import yfinance as yf
from typing import Any
import rerun as rr
import rerun.blueprint as rrb
```
Add helper functions for styling:
```python
brand_colors = {
"AAPL": 0xA2AAADFF,
"AMZN": 0xFF9900FF,
"GOOGL": 0x34A853FF,
"META": 0x0081FBFF,
"MSFT": 0xF14F21FF,
}
def style_plot(symbol: str) -> rr.SeriesLine:
return rr.SeriesLine(
color=brand_colors[symbol],
name=symbol,
)
def style_peak(symbol: str) -> rr.SeriesPoint:
return rr.SeriesPoint(
color=0xFF0000FF,
name=f"{symbol} (peak)",
marker="Up",
)
def info_card(
shortName: str,
industry: str,
marketCap: int,
totalRevenue: int,
**args: dict[str, Any],
) -> rr.TextDocument:
markdown = f"""
- **Name**: {shortName}
- **Industry**: {industry}
- **Market cap**: ${humanize.intword(marketCap)}
- **Total Revenue**: ${humanize.intword(totalRevenue)}
"""
return rr.TextDocument(markdown, media_type=rr.MediaType.MARKDOWN)
```
Add the main function that logs data:
```python
def main() -> None:
symbols = ["AAPL", "AMZN", "GOOGL", "META", "MSFT"]
# Use eastern time for market hours
et_timezone = pytz.timezone("America/New_York")
start_date = dt.date(2024, 3, 18)
dates = [start_date + dt.timedelta(days=i) for i in range(5)]
# Initialize Rerun and spawn a new viewer
rr.init("rerun_example_blueprint_stocks", spawn=True)
# This is where we will edit the blueprint
blueprint = None
# rr.send_blueprint(blueprint)
# Log the stock data for each symbol and date
for symbol in symbols:
stock = yf.Ticker(symbol)
# Log the stock info document as static
rr.log(f"stocks/{symbol}/info", info_card(**stock.info), static=True)
for day in dates:
# Log the styling data as static
rr.log(f"stocks/{symbol}/{day}", style_plot(symbol), static=True)
rr.log(f"stocks/{symbol}/peaks/{day}", style_peak(symbol), static=True)
# Query the stock data during market hours
open_time = dt.datetime.combine(day, dt.time(9, 30), et_timezone)
close_time = dt.datetime.combine(day, dt.time(16, 00), et_timezone)
hist = stock.history(start=open_time, end=close_time, interval="5m")
# Offset the index to be in seconds since the market open
hist.index = hist.index - open_time
peak = hist.High.idxmax()
# Log the stock state over the course of the day
for row in hist.itertuples():
rr.set_time("time", duration=row.Index)
rr.log(f"stocks/{symbol}/{day}", rr.Scalars(row.High))
if row.Index == peak:
rr.log(f"stocks/{symbol}/peaks/{day}", rr.Scalars(row.High))
if __name__ == "__main__":
main()
```
Run the script:
```bash
python stocks.py
```
Without a blueprint, the heuristic layout may not be ideal:
### Creating a simple view
Replace the blueprint section with:
```python
# Create a single chart for all the AAPL data:
blueprint = rrb.Blueprint(
rrb.TimeSeriesView(name="AAPL", origin="/stocks/AAPL"),
)
rr.send_blueprint(blueprint)
```
The `origin` parameter scopes the view to a specific subtree. Now you'll see just the AAPL data:
### Controlling panel state
You can control which panels are visible:
```python
# Create a single chart and collapse the selection and time panels:
blueprint = rrb.Blueprint(
rrb.TimeSeriesView(name="AAPL", origin="/stocks/AAPL"),
rrb.BlueprintPanel(state="expanded"),
rrb.SelectionPanel(state="collapsed"),
rrb.TimePanel(state="collapsed"),
)
rr.send_blueprint(blueprint)
```
### Combining multiple views
Use containers to combine multiple views. The `Vertical` container stacks views, and `row_shares` controls relative sizing:
```python
# Create a vertical layout of an info document and a time series chart
blueprint = rrb.Blueprint(
rrb.Vertical(
rrb.TextDocumentView(name="Info", origin="/stocks/AAPL/info"),
rrb.TimeSeriesView(name="Chart", origin="/stocks/AAPL"),
row_shares=[1, 4],
),
rrb.BlueprintPanel(state="expanded"),
rrb.SelectionPanel(state="collapsed"),
rrb.TimePanel(state="collapsed"),
)
rr.send_blueprint(blueprint)
```
### Specifying view contents
The `contents` parameter provides fine-grained control over what appears in a view. You can include data from multiple sources:
```python
# Create a view with two stock time series
blueprint = rrb.Blueprint(
rrb.TimeSeriesView(
name="META vs MSFT",
contents=[
"+ /stocks/META/2024-03-19",
"+ /stocks/MSFT/2024-03-19",
],
),
rrb.BlueprintPanel(state="expanded"),
rrb.SelectionPanel(state="collapsed"),
rrb.TimePanel(state="collapsed"),
)
rr.send_blueprint(blueprint)
```
### Filtering with expressions
Content expressions can include or exclude subtrees using wildcards. They can reference `$origin` and use `/**` to match entire subtrees:
```python
# Create a chart for AAPL and filter out the peaks:
blueprint = rrb.Blueprint(
rrb.TimeSeriesView(
name="AAPL",
origin="/stocks/AAPL",
contents=[
"+ $origin/**",
"- $origin/peaks/**",
],
),
rrb.BlueprintPanel(state="expanded"),
rrb.SelectionPanel(state="collapsed"),
rrb.TimePanel(state="collapsed"),
)
rr.send_blueprint(blueprint)
```
See [Entity Queries](https://rerun.io/docs/concepts/visualization/entity-queries.md) for complete expression syntax.
### Programmatic layout generation
Since blueprints are Python code, you can generate them dynamically. This example creates a grid with one row per stock symbol:
```python
# Iterate over all symbols and days to create a comprehensive grid
blueprint = rrb.Blueprint(
rrb.Vertical(
contents=[
rrb.Horizontal(
contents=[
rrb.TextDocumentView(
name=f"{symbol}",
origin=f"/stocks/{symbol}/info",
),
]
+ [
rrb.TimeSeriesView(
name=f"{day}",
origin=f"/stocks/{symbol}/{day}",
)
for day in dates
],
name=symbol,
)
for symbol in symbols
]
),
rrb.BlueprintPanel(state="expanded"),
rrb.SelectionPanel(state="collapsed"),
rrb.TimePanel(state="collapsed"),
)
rr.send_blueprint(blueprint)
```
### Saving blueprints from code
You can save programmatically-created blueprints to `.rbl` files:
```python
"""Craft a blueprint with the python API and save it to file."""
import sys
import rerun.blueprint as rrb
path_to_rbl = sys.argv[1]
blueprint = rrb.Blueprint(
rrb.TimeSeriesView(name="AAPL", origin="/stocks/AAPL"),
)
# Save to a file
blueprint.save("rerun_example_blueprint_stocks", path_to_rbl)
```
#### Loading blueprints from any language
Existing blueprint files (e.g. created with the Python SDK or saved from the viewer) can be programmatically loaded into Rerun
This is particularly useful when using Rust or C++ SDKs, since the blueprint API is not yet available for these languages:
```python
"""
Query and display the first 10 rows of a recording in a dataframe view.
The blueprint is being loaded from an existing blueprint recording file.
"""
# python dataframe_view_query_external.py /tmp/dna.rrd /tmp/dna.rbl
import sys
import rerun as rr
path_to_rrd = sys.argv[1]
path_to_rbl = sys.argv[2]
rr.init("rerun_example_dataframe_view_query_external", spawn=True)
rr.log_file_from_path(path_to_rrd)
rr.log_file_from_path(path_to_rbl)
```
This works using the `log_file_from_path` API, which allows you to log any file that contains data that Rerun understands β in this case, blueprint data.
API reference:
- [π Python `log_file_from_path`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log_file_from_path)
- [π¦ Rust `log_file_from_path`](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.log_file_from_path)
- [π C++ `log_file_from_path`](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a20798d7ea74cce5c8174e5cacd0a2c47)
See the [Blueprint API Reference](https://ref.rerun.io/docs/python/stable/common/blueprint_apis/) for complete details.
### Advanced customization
Blueprints support deep customization of view properties. For example:
```python
# Configure a 3D view with custom camera settings
rrb.Spatial3DView(
name="Robot view",
origin="/world/robot",
background=[100, 149, 237], # Light blue
eye_controls=rrb.EyeControls3D(
kind=rrb.Eye3DKind.FirstPerson,
speed=20.0,
),
)
# Configure a time series view with custom axis and time ranges
rrb.TimeSeriesView(
name="Sensor Data",
origin="/sensors",
axis_y=rrb.ScalarAxis(range=(-10.0, 10.0), zoom_lock=True),
plot_legend=rrb.PlotLegend(visible=False),
time_ranges=[
rrb.VisibleTimeRange(
"time",
start=rrb.TimeRangeBoundary.cursor_relative(seq=-100),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
],
)
```
See [Visualizers and Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) for information on overriding component values and controlling visualizers from code.
---
## Youtube overview
While some people might want to read through the documentation on this page, others might prefer to watch a video! If you would like to follow along with the Youtube video, you can find the code used in the video below.
VIDEO
```python
from __future__ import annotations
import math
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
from numpy.random import default_rng
rr.init("rerun_blueprint_example", spawn=True)
rr.set_time("time", sequence=0)
rr.log("log/status", rr.TextLog("Application started.", level=rr.TextLogLevel.INFO))
rr.set_time("time", sequence=5)
rr.log("log/other", rr.TextLog("A warning.", level=rr.TextLogLevel.WARN))
for i in range(10):
rr.set_time("time", sequence=i)
rr.log("log/status", rr.TextLog(f"Processing item {i}.", level=rr.TextLogLevel.INFO))
# Create a text view that displays all logs.
blueprint = rrb.Blueprint(
rrb.TextLogView(origin="/log", name="Text Logs"),
rrb.SelectionPanel(state="expanded"),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
input("Press Enter to continueβ¦")
# Create a spiral of points:
n = 150
angle = np.linspace(0, 10 * np.pi, n)
spiral_radius = np.linspace(0.0, 3.0, n) ** 2
positions = np.column_stack((np.cos(angle) * spiral_radius, np.sin(angle) * spiral_radius))
colors = np.dstack((np.linspace(255, 255, n), np.linspace(255, 0, n), np.linspace(0, 255, n)))[0].astype(int)
radii = np.linspace(0.01, 0.7, n)
rr.log("points", rr.Points2D(positions, colors=colors, radii=radii))
# Create a Spatial2D view to display the points.
blueprint = rrb.Blueprint(
rrb.Spatial2DView(
origin="/",
name="2D Scene",
# Set the background color
background=[105, 20, 105],
# Note that this range is smaller than the range of the points,
# so some points will not be visible.
visual_bounds=rrb.VisualBounds2D(x_range=[-5, 5], y_range=[-5, 5]),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
input("Press Enter to continueβ¦")
rr.log(
"points",
rr.GeoPoints(
lat_lon=[[47.6344, 19.1397], [47.6334, 19.1399]],
radii=rr.Radius.ui_points(20.0),
),
)
# Create a map view to display the chart.
blueprint = rrb.Blueprint(
rrb.MapView(
origin="points",
name="MapView",
zoom=16.0,
background=rrb.MapProvider.OpenStreetMap,
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
input("Press Enter to continueβ¦")
blueprint = rrb.Blueprint(
rrb.Grid(
rrb.MapView(
origin="points",
name="MapView",
zoom=16.0,
background=rrb.MapProvider.OpenStreetMap,
),
rrb.Spatial2DView(
origin="/",
name="2D Scene",
# Set the background color
background=[105, 20, 105],
# Note that this range is smaller than the range of the points,
# so some points will not be visible.
visual_bounds=rrb.VisualBounds2D(x_range=[-5, 5], y_range=[-5, 5]),
),
rrb.TextLogView(origin="/log", name="Text Logs"),
),
rrb.TimePanel(state="expanded"),
rrb.BlueprintPanel(state="expanded"),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
blueprint.save("my_favorite_blueprint", "data/blueprint.rbl")
input("Press Enter to continueβ¦")
rr.log("bar_chart", rr.BarChart([8, 4, 0, 9, 1, 4, 1, 6, 9, 0]))
rng = default_rng(12345)
positions = rng.uniform(-5, 5, size=[50, 3])
colors = rng.uniform(0, 255, size=[50, 3])
radii = rng.uniform(0.1, 0.5, size=[50])
rr.log("3dpoints", rr.Points3D(positions, colors=colors, radii=radii))
tensor = np.random.randint(0, 256, (32, 240, 320, 3), dtype=np.uint8)
rr.log("tensor", rr.Tensor(tensor, dim_names=("batch", "x", "y", "channel")))
rr.log(
"markdown",
rr.TextDocument(
"""
# Hello Markdown!
[Click here to see the raw text](recording://markdown:Text).
"""
),
)
rr.log("trig/sin", rr.SeriesLines(colors=[255, 0, 0], names="sin(0.01t)"), static=True)
for t in range(int(math.pi * 4 * 100.0)):
rr.set_time("time", sequence=t)
rr.set_time("timeline1", duration=t)
rr.log("trig/sin", rr.Scalars(math.sin(float(t) / 100.0)))
blueprint = rrb.Blueprint(
rrb.Grid(
rrb.MapView(
origin="points",
name="MapView",
zoom=16.0,
background=rrb.MapProvider.OpenStreetMap,
),
rrb.Spatial2DView(
origin="/",
name="2D Scene",
# Set the background color
background=[105, 20, 105],
# Note that this range is smaller than the range of the points,
# so some points will not be visible.
visual_bounds=rrb.VisualBounds2D(x_range=[-5, 5], y_range=[-5, 5]),
),
rrb.TextLogView(origin="/log", name="Text Logs"),
rrb.BarChartView(origin="bar_chart", name="Bar Chart"),
rrb.Spatial3DView(
origin="/3dpoints",
name="3D Scene",
# Set the background color to light blue.
background=[100, 149, 237],
# Configure the eye controls.
eye_controls=rrb.EyeControls3D(
kind=rrb.Eye3DKind.FirstPerson,
speed=20.0,
),
),
rrb.TensorView(
origin="tensor",
name="Tensor",
# Explicitly pick which dimensions to show.
slice_selection=rrb.TensorSliceSelection(
# Use the first dimension as width.
width=1,
# Use the second dimension as height and invert it.
height=rr.TensorDimensionSelection(dimension=2, invert=True),
# Set which indices to show for the other dimensions.
indices=[
rr.TensorDimensionIndexSelection(dimension=2, index=4),
rr.TensorDimensionIndexSelection(dimension=3, index=5),
],
# Show a slider for dimension 2 only. If not specified, all dimensions in `indices` will have sliders.
slider=[2],
),
# Set a scalar mapping with a custom colormap, gamma and magnification filter.
scalar_mapping=rrb.TensorScalarMapping(colormap="turbo", gamma=1.5, mag_filter="linear"),
# Fill the view, ignoring aspect ratio.
view_fit="fill",
),
rrb.TextDocumentView(origin="markdown", name="Markdown example"),
rrb.TimeSeriesView(
origin="/trig",
# Set a custom Y axis.
axis_y=rrb.ScalarAxis(range=(-1.0, 1.0), zoom_lock=True),
# Configure the legend.
plot_legend=rrb.PlotLegend(visible=False),
# Set time different time ranges for different timelines.
time_ranges=[
# Sliding window depending on the time cursor for the first timeline.
rrb.VisibleTimeRange(
"time",
start=rrb.TimeRangeBoundary.cursor_relative(seq=-100),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
# Time range from some point to the end of the timeline for the second timeline.
rrb.VisibleTimeRange(
"timeline1",
start=rrb.TimeRangeBoundary.absolute(seconds=300.0),
end=rrb.TimeRangeBoundary.infinite(),
),
],
),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
# Plot any scalar
Rerun can plot numerical data as a time series, even data that wasn't logged with Rerun semantics.
By remapping where a visualizer reads its inputs from, you can separate how you _model_ your data from how you _visualize_ it.
This is useful for plotting custom messages from MCAPs, or data logged via `AnyValues` and `DynamicArchetype`.
As a bonus, logging multiple scalars to the same entity can drastically reduce `.rrd` file sizes.
Each visualizer takes components as input and determines their values from various sources.
By configuring _component mappings_, you can control exactly where each input comes from.
The supported data types are:
- `Float32` and `Float64`
- `Int8`, `Int16`, `Int32`, and `Int64`
- `UInt8`, `UInt16`, `UInt32`, and `UInt64`
- `Boolean`
- Any of the above nested inside of [Arrow structs](https://arrow.apache.org/docs/format/Intro.html#struct).
For background on how visualizers resolve component values, see [Customize views](https://rerun.io/docs/concepts/visualization/customize-views.md).
## Logging custom data
Use `DynamicArchetype` to send data with custom component names alongside regular Rerun data.
Flat arrays and Arrow `StructArray`s are both supported.
This is what the data looks like for the `/plot` entity:
```python
# Custom scalar batch with a cos using a custom component name.
*rr.DynamicArchetype.columns(
archetype="custom",
components={"my_custom_scalar": np.cos(times / 10.0)},
),
# Nested custom scalar batch with a sigmoid inside a struct.
*rr.DynamicArchetype.columns(
archetype="custom",
components={"my_nested_scalar": make_sigmoid_struct_array(64)},
),
```
## Remapping components
A visualizer can source its inputs from any component with a compatible datatype.
For example, the `SeriesLines` visualizer accepts any numerical data for its `Scalar` input.
This works with data from MCAP files, `AnyValues`, or `DynamicArchetype`.
Optional components like `Names` and `Colors` can be sourced similarly from arbitrary data.
The following remaps the `Scalars:scalars` input to read from `custom:my_custom_scalar` instead:
```python
# Green cosine:
# * source scalars from the custom component
# "custom:my_custom_scalar"
# * set the name via an override
# * everything else uses the automatic component mappings,
# so it will pick up colors from the view default.
rr.SeriesLines(names="cosine (custom)").visualizer(
mappings=[
# Map scalars to the custom component.
VisualizerComponentMapping(
target="Scalars:scalars",
source_kind=ComponentSourceKind.SourceComponent,
# Map from custom component
source_component="custom:my_custom_scalar",
),
]
),
```
### Add data by dragging components
You can set up this mapping interactively instead of via the blueprint API: drag a component from the streams tree onto a time series view. If the component has a compatible (numeric) datatype, a new `SeriesLines` visualizer is added that remaps `Scalars:scalars` from it. Non-numeric components (e.g. a string) are rejected, as is dropping a component that the view already plots.
## Selectors for nested data
When your data lives inside an Arrow `StructArray`, use a _selector_ to extract a specific field.
Selectors use a `jq`-inspired syntax (e.g. `.values` to select the `values` field).
Data types are automatically cast when compatible. For example, `Float32` data will be cast to `Float64` as needed by the visualizer.
Here is how to create a nested `StructArray`:
```python
def make_sigmoid_struct_array(steps: int) -> pa.StructArray:
"""Creates a StructArray with a `values` field containing sigmoid data.
Note: We intentionally use float32 here to demonstrate that the data will
be automatically cast to the correct type (float64) when resolved by the
visualizer.
"""
x = np.arange(steps, dtype=np.float32) / 10.0
sigmoid_values = 1.0 / (1.0 + np.exp(-(x - 3.0)))
return pa.StructArray.from_arrays(
[pa.array(sigmoid_values, type=pa.float32())], names=["values"]
)
```
The following remaps the `Scalars:scalars` input to read from `custom:my_nested_scalar` and selects the `values` field:
```python
# Blue sigmoid:
# * source scalars from a nested struct using a selector to
# extract the "values" field
# * set the name and an explicit blue color via overrides
rr.SeriesLines(
names="sigmoid (nested)", colors=[0, 0, 255]
).visualizer(
mappings=[
VisualizerComponentMapping(
target="Scalars:scalars",
source_kind=ComponentSourceKind.SourceComponent,
source_component="custom:my_nested_scalar",
selector=".values",
),
]
),
```
## Providing default values
You can also force a visualizer to use a specific source kind. Setting the source to `Default` makes the visualizer
ignore any store data and use the view's default instead:
```python
# Red sine:
# * set the name via an override
# * explicitly use the view's default for color
# * everything else uses the automatic component mappings,
# so it will pick up scalars from the store.
rr.SeriesLines(names="sine (store)").visualizer(
mappings=[
VisualizerComponentMapping(
target="SeriesLines:colors",
source_kind=ComponentSourceKind.Default,
),
]
),
```
## Full example
The complete example logs three series to a single entity and configures each with a different component mapping strategy. This leads to the following visualizers for the `/plot` entity:
* π [Python](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/howto/component_mapping.py)
* π¦ [Rust](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/howto/component_mapping.rs)
When the view is selected, the selection panel shows an overview of all configured visualizers:
# Visualize fixed-window plots
As of Rerun 0.16, the [TimeSeriesView](https://rerun.io/docs/reference/types/views/time_series_view.md) now supports direct
manipulation of the visible time range. This allows you to create a plot that only shows a fixed window of data.
## VisibleTimeRange
To specify the visible time range, you must pass one or more `VisibleTimeRange` objects to the `time_ranges` parameter of the `TimeSeriesView` blueprint type. If your app only uses a single timeline, you can directly pass a single `VisibleTimeRange` object instead of wrapping it in a list.
The `VisibleTimeRange` object takes three parameters:
- `timeline`: The timeline that the range will apply to. This must match the timeline used to log your data, or if you are only using the rerun-provided timestamps, you can use the strings `"log_time"`, or `"log_tick"`.
- `start`: The start of the visible time range.
- `end`: The end of the visible time range.
The `start` and `end` parameters are set using a `TimeRangeBoundary`:
- To specify an absolute time, you can use the `TimeRangeBoundary.absolute()` method.
- To specify a cursor-relative time, you can use the `TimeRangeBoundary.cursor_relative()` method.
- You can also specify `TimeRangeBoundary.infinite()` to indicate that the start or end of the time range should be unbounded.
In order to account for the different types of timeline (temporal or sequence-based), both the
`TimeRangeBoundary.absolute()` and `TimeRangeBoundary.cursor_relative()` methods can be specified using one of
the keyword args:
- `seconds`/`nanos`: Use these if you called `rr.set_time()` to update the timeline.
- `seq`: Use this if you called `rr.set_time_sequence()` to update the timeline.
## Example syntax
To create a trailing 5 second window plot, you can specify your `TimeSeriesView` like this:
```python
rrb.TimeSeriesView(
origin="plot_path",
time_ranges=rrb.VisibleTimeRange(
timeline="time",
start=rrb.TimeRangeBoundary.cursor_relative(seconds=-5.0),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
)
```
## Full example
For a complete working example, you can run the following code:
```python
#!/usr/bin/env python3
"""A live plot of a random walk using a scrolling fixed window size."""
from __future__ import annotations
import time
import numpy as np
import rerun as rr # pip install rerun-sdk
import rerun.blueprint as rrb
rr.init("rerun_example_fixed_window_plot", spawn=True)
rr.send_blueprint(
rrb.TimeSeriesView(
origin="random_walk",
time_ranges=rrb.VisibleTimeRange(
"time",
start=rrb.TimeRangeBoundary.cursor_relative(seconds=-5.0),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
),
)
cur_time = time.time()
value = 0.0
while True:
cur_time += 0.01
sleep_for = cur_time - time.time()
if sleep_for > 0:
time.sleep(sleep_for)
value += np.random.normal()
rr.set_time("time", timestamp=cur_time)
rr.log("random_walk", rr.Scalars(value))
```
This should create a plot that only shows the last 5 seconds of data. If you select the view, you should
see that the time range is configured as expected.
Alternatively, you can check out a more full-featured example with multiple plot windows [here](https://github.com/rerun-io/rerun/tree/latest/examples/python/live_scrolling_plot).
## Additional notes
- Any time you log data, it has two timepoints associated with it: "log_time", and "log_tick".
# Component mappings
By default, each visualizer reads its input components from the - for example, the `Points3D` visualizer reads colors from `Points3D:colors`.
**Component mappings** let you override this, redirecting any visualizer input to a different component on the same entity. This makes it possible to store multiple variants of the same data and switch between them per view.
To learn more about how visualizers are set up in general, also have a look at the [concept page on customizing Views](https://rerun.io/docs/concepts/visualization/customize-views).
This guide uses a point cloud with two color sets as a running example, but the same technique works for any component!
## Full example
You can find the full example here:
* π [Python](https://github.com/rerun-io/rerun/blob/latest/docs/snippets/all/howto/dual_color_point_cloud.py)
* π¦ [Rust](https://github.com/rerun-io/rerun/blob/latest/docs/snippets/all/howto/dual_color_point_cloud.rs)
## How it works
### Storing multiple component variants with custom archetypes
A standard archetype like `Points3D` assigns a fixed column name, as well as component type & archetype metainformation, to every component it logs (`Points3D:colors`, `Points3D:positions`, etc.).
To store additional variants of a component on the same entity, log them as **custom archetypes**. The easiest way to do this is to use the `DynamicArchetype` utility.
```python
# --- Log positions once, then each color set as a separate custom archetype ---
rr.log(
"pointcloud",
rr.Points3D(positions, radii=0.06),
rr.DynamicArchetype(
"HeightColors",
components={"colors": rr.components.ColorBatch(height_rgba)},
),
rr.DynamicArchetype(
"SpinColors", components={"colors": rr.components.ColorBatch(spin_rgba)}
),
)
```
After this call, the entity's component list contains four components:
- `Points3D:positions`, `Points3D:radii` β from the standard archetype
- `HeightColors:colors` β the first custom variant
- `SpinColors:colors` β the second custom variant
You can inspect this in the viewer by selecting the entity in the streams panel:
### Configuring component mappings in blueprint
To make a visualizer read from an arbitrary source, we need to explicitly set the **component mapping** for a visualizer.
The mapping specifies a *target* (the component the visualizer expects) and a *source* (the component to actually read from).
Everything that isn't explicitly mapped keeps its default behavior.
```python
# --- Blueprint: two side-by-side 3D views with different color mappings ---
blueprint = rrb.Blueprint(
rrb.Horizontal(
rrb.Spatial3DView(
name="Height Colors",
origin="/",
overrides={
"pointcloud": [
rr.Points3D.from_fields().visualizer(
mappings=[
VisualizerComponentMapping(
target="Points3D:colors",
source_kind=ComponentSourceKind.SourceComponent,
source_component="HeightColors:colors",
),
]
),
],
},
),
rrb.Spatial3DView(
name="Spin Colors",
origin="/",
overrides={
"pointcloud": [
rr.Points3D.from_fields().visualizer(
mappings=[
VisualizerComponentMapping(
target="Points3D:colors",
source_kind=ComponentSourceKind.SourceComponent,
source_component="SpinColors:colors",
),
]
),
],
},
),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
In this example, each view overrides only the color source for the `Points3D` visualizer - positions, radii, and everything else are still read from their default sources automatically.
In the viewer you can access and this in the visualizer settings, presented when selecting an entity in a view:
### Configuring component mappings in the UI
You can set up the same mappings interactively without writing any blueprint code:
- Add two 3D views (or clone an existing one)
- **Select the entity** in one of the views.
- In the selection panel, find the visualizer section (e.g. **Points3D**).
- Click on the component row you want to remap (e.g. **colors**) to expand the component mapping options.
- Change the source from the default to the desired custom archetype component (e.g. `HeightColors:colors`).
# Visualize geospatial data
Rerun 0.20 introduced a new [map view](https://rerun.io/docs/reference/types/views/map_view.md).
This guide provides a short overview on how to use it to visualize geospatial data.
## Coordinate system
The map view uses the [ESPG:3857](https://epsg.io/3857) [spherical mercator projection](https://en.wikipedia.org/wiki/Web_Mercator_projection) commonly used by web services such as [OpenStreetMap](https://www.openstreetmap.org/).
This enables the use of commonly available web tiles for the background map.
To be compatible with this view, geospatial data must be expressed using [ESPG:4326](https://epsg.io/4326) (aka WGS84) latitudes and longitudes.
This corresponds to what is commonly referred to as "GPS coordinates."
Rerun provides a set of archetypes prefixed with `Geo` designed to encapsulate such data.
For example, [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md) represent a single geospatial location (or a batch thereof). The location of the Eiffel Tower can be logged as follows:
```python
rr.log("eiffel_tower", rr.GeoPoints(lat_lon=[48.858222, 2.2945]))
```
Both the latitude and longitude must be provided in degrees, with positive values corresponding to the North, resp. East directions.
Note that Rerun always expects latitudes first and longitudes second.
As there is [no accepted ordering standard](https://stackoverflow.com/questions/7309121/preferred-order-of-writing-latitude-longitude-tuples-in-gis-services), our APIs strive to make this ordering choice as explicit as possible.
In this case, the `lat_lon` argument is keyword-only and must thus be explicitly named as a reminder of this order.
## Types of geometries
Rerun currently supports two types of geometries:
- [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md): batch of individual points, with optional [radius](https://rerun.io/docs/reference/types/components/radius.md) and [color](https://rerun.io/docs/reference/types/components/color.md)
- [`GeoLineStrings`](https://rerun.io/docs/reference/types/archetypes/geo_line_strings.md): batch of line strings, with optional [radius](https://rerun.io/docs/reference/types/components/radius.md) and [color](https://rerun.io/docs/reference/types/components/color.md)
> [!NOTE]
> Polygons are planned but are not supported yet (see [this issue](https://github.com/rerun-io/rerun/issues/8066)).
As in other views, radii may be expressed either as UI points (negative values) or scene units (positive values).
For the latter case, the map view uses meters are scene units.
Apart from the use of latitude and longitude, `GeoPoints` and `GeoLineStrings` are otherwise similar to the [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md) and [`LineStrip2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md) archetypes used in the [2D view](https://rerun.io/docs/reference/types/views/spatial2d_view.md).
The map view supports several types of background maps, including a few from [Mapbox](https://www.mapbox.com).
A Mapbox access token is required to use them.
It must be provided either using the `RERUN_MAPBOX_ACCESS_TOKEN` environment variable or configured in the settings screen ("Settingsβ¦" item in the Rerun menu).
An access token may be freely obtained by creating a Mapbox account.
## Creating a map view from code
Like other views, the map view can be configured using the [blueprint API](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md):
```python
import rerun.blueprint as rrb
blueprint = rrb.Blueprint(
rrb.MapView(
origin="/robot/position",
name="map view",
zoom=16.0,
background=rrb.MapProvider.OpenStreetMap,
),
)
```
Check the [map view](https://rerun.io/docs/reference/types/views/map_view.md) reference for details.
# Use multiple native viewers
You can run multiple Native Viewer windows simultaneously, each displaying different data or views of the same data.
## How it works
Every Native Viewer binds to a gRPC port on startup. By default, this is port `9876`. When you run `rerun`, it checks if a viewer is already listening on that port:
- **If yes**: it connects to the existing viewer (or sends data to it)
- **If no**: it starts a new viewer on that port
To open multiple viewer windows, use different ports with the `--port` flag.
## Examples
```sh
# Start a viewer on the default port (9876)
$ rerun &
# This does nothing β a viewer is already running on :9876
$ rerun &
# Start a second viewer on port 6789
$ rerun --port 6789 &
# Log an image to the first viewer (port 9876)
$ rerun image.jpg
# Log an image to the second viewer (port 6789)
$ rerun --port 6789 image.jpg
```
## From the SDK
When using `connect_grpc()` from the SDK, specify the port to target a specific viewer:
```python
import rerun as rr
# Connect to viewer on default port
rr.init("rerun_example_demo")
rr.connect_grpc()
# Or connect to a specific port
rr.connect_grpc("rerun+http://127.0.0.1:6789")
```
## Tips
- Use `spawn()` to automatically start a new viewer if needed β it will reuse an existing viewer on the default port if one is running
- Each viewer maintains its own Chunk Store, so data sent to different viewers is independent
- The Web Viewer doesn't use gRPC ports the same way β it connects via WebSocket when served locally
# Limit the viewer's memory usage
### --memory-limit
The Rerun Viewer can not yet view more data than fits in RAM. The more data you log, the more RAM the Rerun Viewer will use. When it reaches a certain limit, the oldest data will be dropped. The default limit is to use up to 75% of the total system RAM.
You can set the limit with the `--memory-limit` command-line argument, or the `memory_limit` argument of [`rr.spawn`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.spawn).
Alternatively, you can adjust the limit for an active session also in the viewer's settings. It will be reset to the default the next time you open the viewer.
# Visualize state changes
The [StateTimelineView](https://rerun.io/docs/reference/types/views/state_timeline_view.md) shows how entities transition between discrete states over time. Each entity becomes a horizontal lane, and each logged state is rendered as a colored band that runs until the next change. This is a good fit for state machines, mode transitions, sensor health, or any other piece of data that's better described as "what state am I in right now?" than as a numerical value.
## Logging state changes
Use [`StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change.md) to log a transition. Each call marks the start of a new state at the current time; the previous state implicitly ends. The state value is a string, so you can use any label that's meaningful for your application.
```python
# Log state transitions for two entities. Each call marks the start of a new
# state; the previous state implicitly ends. The `/door` lane uses the
# `StateConfiguration` above, while `/window` gets default styling (raw value
# as label, hashed color).
rr.set_time("step", sequence=0)
rr.log("door", rr.StateChange(state="open"))
rr.log("window", rr.StateChange(state="closed"))
rr.set_time("step", sequence=1)
rr.log("door", rr.StateChange(state="closed"))
rr.set_time("step", sequence=3)
rr.log("window", rr.StateChange(state="open"))
rr.set_time("step", sequence=4)
rr.log("door", rr.StateChange(state="open"))
```
### Notes
- The view groups state changes by entity path, so logging to `/door` and `/window` produces two separate lanes.
- Logging the same state value twice in a row is a no-op for visualization, only transitions to a different value start a new phase.
- Each phase runs from its `StateChange` time to the next `StateChange` time on the same entity. The final phase extends indefinitely.
## Customizing labels, colors, and visibility
To override the default styling, log a [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md) to the same entity. `values`, `labels`, `colors`, and `visible` are parallel arrays β index `i` of each describes the same state value. Anything you don't provide falls back to the default (raw value as label, hashed color, visible).
It is usually best to log `StateConfiguration` as static, since it describes how to display values rather than a moment in time.
```python
# Customize how each state value is displayed (label, color, visibility).
# Log as static so the configuration applies for the entire recording.
rr.log(
"door",
rr.StateConfiguration(
values=["open", "closed"],
labels=["Open", "Closed"],
colors=[0x4CAF50FF, 0xEF5350FF],
),
static=True,
)
```
## Visualize any component as state
You don't have to log [`StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change.md) to use this view. Any component whose data is string-, boolean-, or number-like can drive a lane by **remapping** the visualizer's `StateChange:state` input to read from it instead. This lets you separate how you _model_ your data from how you _visualize_ it. For example, visualizing a robot mode that you logged as a plain string via `AnyValues` or `DynamicArchetype` (the same idea as [Plot any scalar](https://rerun.io/docs/howto/visualization/plot-any-scalar.md), applied to the state slot).
The supported source data types are:
- `Utf8` and `LargeUtf8` (rendered as string states)
- `Boolean` (rendered as two states)
- `Int8`, `Int16`, `Int32`, `Int64`, `UInt8`, `UInt16`, `UInt32`, `UInt64`, `Float16`, `Float32`, and `Float64` (rendered as numeric states)
For background on how visualizers resolve their inputs, see [Component mappings](https://rerun.io/docs/howto/visualization/component-mappings.md) and [Customize views](https://rerun.io/docs/concepts/visualization/customize-views.md).
For example, log a robot mode as a plain string component:
```python
# Log a robot mode as a plain string component β note that this is *not* a
# `StateChange`, just an arbitrary string logged via `AnyValues`. It shows up on
# the entity as the component `AnyValues:mode`.
modes = ["booting", "idle", "driving", "idle", "charging"]
for step, mode in enumerate(modes):
rr.set_time("step", sequence=step)
rr.log("robot", rr.AnyValues(mode=mode))
```
Then point the state-timeline visualizer at it by remapping `StateChange:state`:
```python
# Remap the state-timeline visualizer's `StateChange:state` input to read from
# the custom `AnyValues:mode` component instead of an actual `StateChange`.
# Any string, boolean, or numeric component can be visualized this way.
blueprint = rrb.Blueprint(
rrb.StateTimelineView(
origin="/",
name="Robot mode",
overrides={
"robot": [
rr.StateChange.from_fields().visualizer(
mappings=[
VisualizerComponentMapping(
target="StateChange:state",
source_kind=ComponentSourceKind.SourceComponent,
source_component="AnyValues:mode",
),
],
),
],
},
),
)
rr.send_blueprint(blueprint)
```
### Add data by dragging components
You can set up the same mapping interactively: drag a component from the streams tree onto a State Timeline view. If the component is a compatible source (string, boolean, or numeric), a new lane is added that remaps `StateChange:state` from it. Incompatible components (e.g. a blob or tensor) are rejected, as is dropping a component that the view already visualizes.
## Setting up the view via blueprint
The State Timeline view is also created automatically when `StateChange` data is present, but you can also configure it explicitly via the blueprint API:
```python
# Place a state timeline view at the root. The viewer will create one
# automatically as soon as it sees `StateChange` data, but the blueprint API
# lets you control the origin, name, and layout explicitly.
blueprint = rrb.Blueprint(
rrb.StateTimelineView(origin="/", name="Doors and windows"),
)
rr.send_blueprint(blueprint)
```
# React to events in the Viewer
We support registering callbacks to Viewer events in these environments:
- Web browsers, through our [JS package](https://rerun.io/docs/howto/integrations/embed-web.md)
- Jupyter notebooks, through our [Notebook API](https://rerun.io/docs/howto/integrations/embed-notebooks.md)
For users extending the Viewer through the Rust [`re_viewer`](https://docs.rs/re_viewer/latest/re_viewer/) crate, there are two options:
- Use [`StartupOptions.on_event`](https://docs.rs/re_viewer/latest/re_viewer/struct.StartupOptions.html#structfield.on_event) to register
the same events available on the web and in Jupyter.
- [Extend the UI](https://github.com/rerun-io/rerun/tree/main/examples/rust/custom_callback) to add your own widgets using `egui`, and
fire completely custom events.
# Implement custom visualizations (Rust only)
There are three ways to extend the Rerun Viewer with custom Rust code, depending on how deep you need to go:
embedding custom UI panels alongside the Viewer, adding a custom visualizer to a built-in view, or implementing an entirely new view class.
> [!WARNING]
> The interfaces for extending the Viewer are not yet stable. Expect code implementing custom extensions to break with every release of Rerun.
## Embedding custom UI in the Viewer
In the above screenshot you see the example [`extend_viewer_ui`](https://github.com/rerun-io/rerun/tree/main/examples/rust/extend_viewer_ui), which contains the Rerun Viewer to the left and a custom panel to the right. In this example the panel contains a hierarchical text view of the loaded data.
### How to build it
The Rerun Viewer is defined by the crate [`re_viewer`](https://github.com/rerun-io/rerun/tree/main/crates/viewer/re_viewer). It uses the popular Rust GUI library [`egui`](https://github.com/emilk/egui) (written by our CTO) and its framework [`eframe`](https://github.com/emilk/egui/tree/master/crates/eframe). To extend the UI you need to create your own `eframe` application and embed `re_viewer` inside of it. You can then use `egui` to add custom panels and windows.
The best way to get started is by reading [the source code of the `extend_viewer_ui` example](https://github.com/rerun-io/rerun/tree/main/examples/rust/extend_viewer_ui).
## Custom visualizers for built-in views
You can register custom visualizers with existing built-in views (such as the 3D Spatial View) without having to implement an entire view class from scratch. This is the right approach when your data fits naturally into an existing view but needs custom rendering. A custom visualizer typically consists of a custom archetype, a visualizer system, and optionally a custom GPU renderer.
The [`custom_visualizer`](https://github.com/rerun-io/rerun/tree/main/examples/rust/custom_visualizer) example demonstrates this by adding a GPU-rendered heightfield to the built-in 3D Spatial View. See its README for a detailed walkthrough of the three parts involved.
## Custom view classes
If you need a completely new kind of view (not just a new visualizer within an existing view), you can implement a custom view class.
The [`custom_view`](https://github.com/rerun-io/rerun/tree/main/examples/rust/custom_view) example demonstrates how to add a fully custom View class to Rerun on startup.
Views that are added this way have access to the exact same interfaces as all other Views,
meaning that any of the built-in Views can serve as an additional example on how to implement Views.
The best way to get started is by reading [the source code of the `custom_view` example](https://github.com/rerun-io/rerun/tree/main/examples/rust/custom_view).
## Future work
In the future we'll allow embedding your own GUI widgets inside existing views.
Beyond that we want to open customizability to more languages and support adding ui elements with [callbacks](https://github.com/rerun-io/rerun/issues/2691)
via blueprint definitions.
For more information check https://github.com/rerun-io/rerun/issues/3087
# Dataset Resampling
This snippet demonstrates how to resample a dataset based on the time index of one component
within your data. This is particularly helpful when you have data that is produced at very
different frequencies.
First, load a dataset to use for evaluation:
```python
sample_dataset_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "dataset"
)
server = rr.server.Server(datasets={"dataset": sample_dataset_path})
# Using OSS server for demonstration but in practice replace with
# the URL of your cloud instance
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="dataset")
```
## Investigate time ranges
Before we do the resampling, we can examine the dataset's time ranges using the function
`get_index_ranges()`. This is not strictly necessary for the resampling work to follow, but
it can be helpful during investigation of your data. This will show you the start and
end values for all indexes in your dataset, one per segment.
```python
(
dataset
.get_index_ranges()
.select(
"rerun_segment_id",
"time_1:start",
"time_1:end",
"time_2:start",
"time_2:end",
"time_3:start",
"time_3:end",
)
.sort("rerun_segment_id")
.show()
)
```
## Prior to resampling
The sample data we have loaded is very basic, but it demonstrates having components from
three different entities at different times in the dataset. The code below demonstrates
what the data looks like before resampling. In order to do data analysis on this DataFrame
you would likely need to do some aggregation or window across the time index.
```python
time_index = "time_3"
columns_of_interest = [
"rerun_segment_id",
time_index,
"/obj1:Points3D:positions",
"/obj2:Points3D:positions",
"/obj3:Points3D:positions",
]
(
dataset
.reader(index=time_index)
.select(*columns_of_interest)
.sort("rerun_segment_id", time_index)
.show()
)
# +----------------------------------+--------+--------------------------+--------------------------+--------------------------+
# | rerun_segment_id | time_3 | /obj1:Points3D:positions | /obj2:Points3D:positions | /obj3:Points3D:positions |
# +----------------------------------+--------+--------------------------+--------------------------+--------------------------+
# | 141a866deb2d49f69eb3215e8a404ffc | 1 | [[49.0, 0.0, 0.0]] | [[44.0, 1.0, 0.0]] | [[1.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 2 | [[27.0, 0.0, 0.0]] | [[42.0, 1.0, 0.0]] | |
# | 141a866deb2d49f69eb3215e8a404ffc | 3 | [[25.0, 0.0, 0.0]] | [[30.0, 1.0, 0.0]] | [[3.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 4 | [[38.0, 0.0, 0.0]] | [[19.0, 1.0, 0.0]] | |
# | 141a866deb2d49f69eb3215e8a404ffc | 5 | [[17.0, 0.0, 0.0]] | [[5.0, 1.0, 0.0]] | [[5.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 6 | [[2.0, 0.0, 0.0]] | [[35.0, 1.0, 0.0]] | |
# | 141a866deb2d49f69eb3215e8a404ffc | 7 | [[44.0, 0.0, 0.0]] | [[4.0, 1.0, 0.0]] | [[7.0, 2.0, 0.0]] |
```
## Resampled data
The snippet below demonstrates resampling using two lines. First we create a new DataFrame
which contains the index values we care about per segment. It is *very* important in
doing this that you do not set `fill_latest_at=True`. Otherwise it would negate the effect
we are trying to produce where we only have rows for which we have data in our component
of interest. The required output of this DataFrame is only the segment ID and the index
value.
Once we have a DataFrame with these index values, we can now query the dataset using that
DataFrame. You can see from the output below that we generate one row per time index for
which the component of interest is not null.
```python
resample_column = "/obj3:Points3D:positions"
times_of_interest = (
dataset
.reader(index=time_index)
.filter(col(resample_column).is_not_null())
.select("rerun_segment_id", time_index)
)
(
dataset
.reader(
index=time_index,
using_index_values=times_of_interest,
fill_latest_at=True,
)
.select(*columns_of_interest)
.sort("rerun_segment_id", time_index)
.show()
)
# +----------------------------------+--------+--------------------------+--------------------------+--------------------------+
# | rerun_segment_id | time_3 | /obj1:Points3D:positions | /obj2:Points3D:positions | /obj3:Points3D:positions |
# +----------------------------------+--------+--------------------------+--------------------------+--------------------------+
# | 141a866deb2d49f69eb3215e8a404ffc | 1 | [[49.0, 0.0, 0.0]] | [[44.0, 1.0, 0.0]] | [[1.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 3 | [[25.0, 0.0, 0.0]] | [[30.0, 1.0, 0.0]] | [[3.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 5 | [[17.0, 0.0, 0.0]] | [[5.0, 1.0, 0.0]] | [[5.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 7 | [[44.0, 0.0, 0.0]] | [[4.0, 1.0, 0.0]] | [[7.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 10 | [[12.0, 0.0, 0.0]] | [[6.0, 1.0, 0.0]] | [[10.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 12 | [[13.0, 0.0, 0.0]] | [[17.0, 1.0, 0.0]] | [[12.0, 2.0, 0.0]] |
# | 141a866deb2d49f69eb3215e8a404ffc | 13 | [[20.0, 0.0, 0.0]] | [[32.0, 1.0, 0.0]] | [[13.0, 2.0, 0.0]] |
```
# Creating sub-datasets
When experimenting with new features it's often practical to work with a subset of data without modifying the original.
A sub-dataset references the same underlying RRD files so no data is copied.
The dependencies in this example are contained in `rerun-sdk[all]`.
## Setup
Simplified setup to launch the local server for demonstration.
In practice you'll connect to your cloud instance.
```python
from __future__ import annotations
from pathlib import Path
import pyarrow as pa
import pyarrow.compute as pc
from datafusion import col, lit
from datafusion import functions as F
import rerun as rr
sample_5_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
source_dataset = client.get_dataset(name="sample_dataset")
```
## Helper function
Query the source dataset's [manifest](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) for storage URLs per (segment, layer) pair and re-register them into a new dataset.
```python
def create_sub_dataset(
client: rr.catalog.CatalogClient,
source: rr.catalog.DatasetEntry,
name: str,
segment_ids: list[str],
) -> rr.catalog.DatasetEntry:
"""Create a new dataset with a subset of segments from another dataset."""
# Look up the storage URLs of the selected segments.
selected = pa.table(
source
.segment_table()
.filter(
F.in_list(col("rerun_segment_id"), [lit(s) for s in segment_ids])
)
.select("rerun_storage_urls", "rerun_layer_names")
)
sub_dataset = client.create_dataset(name)
# Flatten the per-segment lists into the (url, layer) pairs to register.
uris = pc.list_flatten(selected.column("rerun_storage_urls")).to_pylist()
layers = pc.list_flatten(selected.column("rerun_layer_names")).to_pylist()
if uris:
sub_dataset.register(uris, layer_name=layers).wait()
return sub_dataset
```
## Selecting segments
Select segments by any criteria β a hardcoded list, a slice, or a filtered query based on segment properties or metadata joins.
```python
# View available segments
print("Available segments:")
print(
source_dataset
.segment_table()
.select("rerun_segment_id")
.sort("rerun_segment_id")
)
# Select a subset β here we pick the first 3 segments.
all_segment_ids = source_dataset.segment_ids()
subset_ids = all_segment_ids[:3]
```
## Creating the sub-dataset
```python
sub_dataset = create_sub_dataset(
client, source_dataset, "my_experiment", subset_ids
)
```
## Verifying the result
```python
print("\nSub-dataset segments:")
print(
sub_dataset
.segment_table()
.select("rerun_segment_id", "rerun_layer_names")
.sort("rerun_segment_id")
)
print("\nSub-dataset storage URLs:")
print(
sub_dataset
.segment_table()
.select("rerun_segment_id", "rerun_layer_names", "rerun_storage_urls")
.sort("rerun_segment_id")
)
```
## Cleanup
Delete the sub-dataset when it is no longer needed.
This only removes the dataset entry from the catalog. The underlying RRD storage is not affected.
```python
# When done experimenting, delete the sub-dataset.
# This only removes the dataset entry β the underlying RRD storage is not
# affected.
sub_dataset.delete()
```
# Common Dataframe Operations
Dataframes are core to modern analytics workflows.
Rerun provides a dataframe interface to your data via [DataFusion](https://datafusion.apache.org/python/).
This example performs a series of joins, filters, etc that highlight a variety of common operations in context.
Because datafusion has a lazy execution model it is generally more performant to use datafusion for processing,
however datafusion does allow conversion to dataframes for popular tools (pandas, polars, pyarrow).
The dependencies in this example are contained in `rerun-sdk[all]`.
## Setup
Perform initial import and spawn local server for demonstration.
In practice you'll connect to your cloud instance.
```python
from __future__ import annotations
from pathlib import Path
import datafusion as dfn
import numpy as np
import pyarrow as pa
from datafusion import col, lit
from datafusion import functions as F
import rerun as rr
sample_5_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="sample_dataset")
observations = dataset.filter_contents(["/observation/**"]).reader(
index="real_time"
)
```
## Group-by / aggregation
Perform an aggregation on the episodes to track the first and last timestamp for the columns of interest.
```python
first_last = observations.aggregate(
col("rerun_segment_id"),
[
F.first_value(col("real_time")).alias("start"),
F.last_value(col("real_time")).alias("end"),
],
)
# Sort for consistency here
first_last = first_last.sort("start")
pa.table(first_last)["start"][0]
```
## Join and query
Some of our columns start much later than others.
Find out how often this delay exceeds some threshold.
> [!WARNING]
> **Performance warning:**
> Even though datafusion pulls data lazily, we don't currently decouple our payload from its timeline.
> E.g. in this example this means that we have to pull the full camera data to inspect their min/max timestamps.
> This works quickly when the data is already local and in memory, but can be a bottleneck on cloud at scale.
```python
joints = dataset.filter_contents(["/observation/joint_positions"])
# Find the earliest joint position in each episode (cast to unix epoch
# nanoseconds for easier math later)
joint_min_t = (
joints
.reader(index="real_time")
.with_column("joint_epoch_ns", col("real_time").cast(pa.int64()))
.select("rerun_segment_id", "joint_epoch_ns")
.aggregate(
col("rerun_segment_id"),
F.min(col("joint_epoch_ns")).alias("joint_min_t"),
)
)
cameras = dataset.filter_contents(["/camera/**"])
# Find the earliest camera frame in each episode (cast to unix epoch
# nanoseconds for easier math later)
camera_min_t = (
cameras
.reader(index="real_time")
.with_column("camera_epoch_ns", col("real_time").cast(pa.int64()))
.select(
"rerun_segment_id",
col("real_time").cast(pa.int64()).alias("camera_epoch_ns"),
)
.aggregate(
col("rerun_segment_id"),
F.min(col("camera_epoch_ns")).alias("camera_min_t"),
)
)
# Join the two dataframes
min_t = camera_min_t.join(
joint_min_t.with_column_renamed("rerun_segment_id", "segment_id"),
left_on="rerun_segment_id",
right_on="segment_id",
how="left",
)
delta_t = min_t.select(
col("rerun_segment_id"),
(col("camera_min_t") - col("joint_min_t")).alias("start_delta_t"),
)
THRESHOLD_S = 1
NANO_S = 1_000_000_000
outliers = delta_t.filter(
dfn.Expr.between(
col("start_delta_t"),
-THRESHOLD_S * NANO_S,
THRESHOLD_S * NANO_S,
negated=True,
),
)
outliers = outliers.with_column(
"start_delta_t_s", col("start_delta_t") / 1_000_000_000.0
)
print(
f"{outliers.count()=}\n",
f"{joint_min_t.count()=}\n",
f"{camera_min_t.count()=}",
sep="",
)
```
## Extract sub-episodes from recording
Oftentimes a recording will capture multiple episodes.
For instance a robotic arm may place multiple items, where each item could be considered an episode.
This example looks for contiguous time ranges where the gripper opens and closes in order to separate these sub-episodes for further downstream processing.
```python
# Grab a dataframe
all_data = (
dataset
.filter_contents(["/action/**", "/observation/**"])
.reader(index="real_time", fill_latest_at=True)
.filter(
col(
"/observation/joint_positions:Scalars:scalars"
).is_not_null() # filter out rows where there is no observation
)
)
# Drop heavy columns for performance
light_slice = all_data.select(
"rerun_segment_id",
"real_time",
"/observation/gripper_position:Scalars:scalars",
)
# Define criteria for sub-episode start/end
THRESHOLD = 0.1
light_slice = light_slice.with_column(
"gripper_open",
col("/observation/gripper_position:Scalars:scalars") > [THRESHOLD],
)
# Find start and end
light_slice = light_slice.with_column(
"prev_gripper_open",
F.lag(
col("gripper_open"),
default_value=False,
partition_by=[col("rerun_segment_id")],
order_by=[col("real_time")],
),
)
light_slice = light_slice.with_column(
"gripper_change",
col("gripper_open").cast(pa.int8())
- col("prev_gripper_open").cast(pa.int8()),
)
slice_times = light_slice.with_column(
"start",
F
.case(col("gripper_change"))
.when(lit(1), col("real_time"))
.otherwise(lit(None)),
).with_column(
"end",
F
.case(col("gripper_change"))
.when(lit(-1), col("real_time"))
.otherwise(lit(None)),
)
# Helper because pyarrow timestamps didn't have a nice min/max utility
max_ts = pa.scalar(np.iinfo(np.int64).max, type=pa.timestamp("ns"))
min_ts = pa.scalar(
np.iinfo(np.int64).min + 1_000_000_000, type=pa.timestamp("ns")
)
# This generates the column for the last observed start time
slice_dense_times = (
slice_times
.select("rerun_segment_id", "real_time", "start", "end")
.with_column(
"dense_start",
F.last_value(col("start")).over(
dfn.expr.Window(
window_frame=dfn.expr.WindowFrame("rows", None, 0),
order_by=col("real_time"),
partition_by=col("rerun_segment_id"),
null_treatment=dfn.common.NullTreatment.IGNORE_NULLS,
)
),
)
.fill_null(value=max_ts, subset=["dense_start"])
)
# This generates the column for the next observed end time (by finding the
# last_value in reversed order)
slice_dense_times = slice_dense_times.with_column(
"dense_end",
F.last_value(col("end")).over(
dfn.expr.Window(
window_frame=dfn.expr.WindowFrame("rows", None, 0),
order_by=col("real_time").sort(ascending=False),
partition_by=col("rerun_segment_id"),
null_treatment=dfn.common.NullTreatment.IGNORE_NULLS,
)
),
).fill_null(value=min_ts, subset=["dense_end"])
slice_dense_times = slice_dense_times.select(
"rerun_segment_id", "real_time", "dense_start", "dense_end"
)
sub_episodes = slice_dense_times.filter(
dfn.Expr.between(col("real_time"), col("dense_start"), col("dense_end")),
)
print(f"{sub_episodes.count()=}")
```
# Query video streams
Video streams provide the best compression ratio for camera feeds, but require special handling when querying data back from a catalog server.
For more details about the different video types we support see our [video reference](https://rerun.io/docs/concepts/logging-and-ingestion/video.md).
This guide focuses on querying [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) data from a catalog server,
including how to decode individual frames and how to export entire streams to MP4 files.
The dependencies in this example require `rerun-sdk[all]` and `av` for video decoding.
## Setup
Simplified setup to launch the local server for demonstration.
In practice you'll connect to your cloud instance.
```python
from fractions import Fraction
from io import BytesIO
from pathlib import Path
import av
import numpy as np
import pyarrow as pa
from datafusion import col
import rerun as rr
sample_video_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "video_sample"
)
server = rr.server.Server(datasets={"video_dataset": sample_video_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="video_dataset")
df = dataset.filter_contents(["/video_stream/**"]).reader(index="log_time")
times = pa.table(df.select("log_time"))["log_time"].to_numpy()
```
## Understanding video stream data
Video streams are logged using the [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) archetype,
which stores encoded video samples (frames) along with codec information.
Key columns you'll work with:
- `VideoStream:codec` - The video codec used (e.g., H.264)
- `VideoStream:sample` - The encoded video frame data (in Annex B format for H.264)
## Checking the video codec
Before processing video data, verify the codec matches what you expect:
```python
codec_column = "/video_stream:VideoStream:codec"
num_codec_matches = df.select(
col(codec_column)[0] == rr.VideoCodec.H264.value
).count()
if num_codec_matches != df.select(codec_column).count():
raise ValueError(
f"Expected H.264 codec {rr.VideoCodec.H264.value}, "
f"got {df.select(codec_column).limit(1)}"
)
```
## Decoding a specific frame
Unlike raw images, video frames are encoded using inter-frame compression.
To decode a specific frame, you must decode from the beginning of the stream (or from the most recent keyframe) and iterate forward.
`av` handles keyframe detection internally during decoding.
```python
video_column = "/video_stream:VideoStream:sample"
selected_frame_index = 3 # Pick an arbitrary frame to decode
# Query all samples up to and including the target frame.
# We need to decode from the start (or a keyframe) to reach our target.
selected_time = times[selected_frame_index]
video_df = df.filter(col("log_time") <= selected_time).select(
"log_time", video_column
)
pa_table = pa.table(video_df)
# Concatenate samples into a byte buffer
samples = pa_table[video_column].to_numpy()
sample_times = pa_table["log_time"].to_numpy()
sample_bytes = b""
for sample in samples:
sample_bytes += sample[0].tobytes()
data_buffer = BytesIO(sample_bytes)
# Decode using PyAV
container = av.open(data_buffer, format="h264", mode="r")
video_stream: av.video.stream.VideoStream = container.streams.video[0]
start_time = sample_times[0]
# Decode all frames up to our target, keeping only the last one
frame = None
for packet, time in zip(
container.demux(video_stream), sample_times, strict=False
):
packet.time_base = Fraction(1, 1_000_000_000) # Timestamps in nanoseconds
packet.pts = int(time - start_time)
packet.dts = packet.pts # No B-frames, so dts == pts
for decoded_frame in packet.decode():
frame = decoded_frame
if not isinstance(frame, av.VideoFrame):
raise RuntimeError("Failed to decode frame.")
image = np.asarray(frame.to_image())
print(f"Decoded frame shape: {image.shape}")
```
## Efficient random access with keyframe information
The example above queries all samples from the start of the stream, which can be inefficient for long videos.
For better performance with random access, you can add keyframe information as a layer.
### Adding keyframe information as a layer
You can analyze your video data once to identify keyframes and register them as a separate layer:
```python
# Preprocessing step: Add keyframe information to existing video data as a layer
# This is typically done once to make subsequent queries faster
# Query all video samples from the existing recording
video_samples_df = df.select("log_time", video_column)
video_table = pa.table(video_samples_df)
sample_times = video_table["log_time"].to_numpy()
samples = video_table[video_column].to_numpy()
# Concatenate all samples to analyze keyframes
sample_bytes = b""
for sample in samples:
sample_bytes += sample[0].tobytes()
# Decode the video to detect keyframes
data_buffer = BytesIO(sample_bytes)
container = av.open(data_buffer, format="h264", mode="r")
video_stream = container.streams.video[0]
# Identify which samples are keyframes
keyframe_times = []
for packet, ts in zip(container.demux(video_stream), sample_times):
if packet.is_keyframe:
keyframe_times.append(ts)
container.close()
keyframe_values = [True] * len(keyframe_times)
print(f"Found {len(keyframe_times)} keyframes")
# Save keyframe data as a separate layer
# Get the segment ID to align with the original recording
segment_ids = dataset.segment_ids()
first_segment_id = segment_ids[0]
# Create time column and content using the columnar API
# Make sure the timeline matches the original video stream
timeline = "log_time"
time_column = rr.TimeColumn(timeline=timeline, timestamp=keyframe_times)
content = rr.DynamicArchetype.columns(
archetype="KeyframeData", components={"is_keyframe": keyframe_values}
)
# Write to a new file as a layer
layer_path = TMP_DIR / "keyframe_layer.rrd"
with rr.RecordingStream(
application_id="keyframes",
recording_id=first_segment_id, # Match original recording_id
) as rec:
rec.save(layer_path)
rec.send_columns("/video_stream", indexes=[time_column], columns=[*content])
# Register the layer with the dataset
dataset.register([layer_path.as_uri()], layer_name="keyframes")
print(f"Registered keyframe layer at {layer_path}")
```
This preprocessing approach:
- Decodes the video once to detect which packets are keyframes using `packet.is_keyframe`
- Creates sparse data containing only keyframe timestamps
- Writes the keyframe data to a separate RRD file
- Registers it as a layer on the dataset
Once registered, the layer data appears as additional columns when querying the dataset (see [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) for details on datasets and layers).
### Querying with keyframe information
With the keyframe layer registered, you can query only the samples between the nearest keyframe and your target frame,
significantly reducing the amount of data to fetch and decode:
```python
# Query using keyframe information for efficient random access
# Assume we've already added keyframe information via the preprocessing step
# above
target_frame_index = 42
target_time = times[target_frame_index]
# Create a reader that includes the keyframe layer data
# The column name follows the pattern: /{entity_path}:{component_name}
keyframe_column = "/video_stream:is_keyframe"
full_df = dataset.filter_contents(["/video_stream/**"]).reader(index="log_time")
# Query to find the most recent keyframe at or before the target time.
# Since we only log when is_keyframe=True, any row with this column present
# is a keyframe
keyframe_slice = full_df.filter(
(col("log_time") <= target_time) & col(keyframe_column).is_not_null()
)
closest_keyframe_df = keyframe_slice.aggregate(
[],
[
F.last_value(col("log_time"), order_by=[col("log_time")]).alias(
"latest_keyframe"
)
],
)
keyframe_result = pa.table(closest_keyframe_df)
# Start decoding from the most recent keyframe
start_time = keyframe_result["latest_keyframe"].to_numpy()[0]
start_frame_idx = np.searchsorted(times, start_time)
frames_saved = target_frame_index - start_frame_idx
print(
f"Found keyframe at frame {start_frame_idx}, "
f"saved decoding {frames_saved} frames"
)
# Query only the video samples from keyframe to target (much more efficient!)
efficient_video_df = df.filter(
col("log_time").between(start_time, target_time)
).select("log_time", video_column)
efficient_table = pa.table(efficient_video_df)
frames_to_decode = len(efficient_table)
print(
f"Decoding {frames_to_decode} frames "
f"(vs {target_frame_index + 1} without keyframe info)"
)
# Now decode just this smaller range
samples = efficient_table[video_column].to_numpy()
sample_times = efficient_table["log_time"].to_numpy()
sample_bytes = b""
for sample in samples:
sample_bytes += sample[0].tobytes()
data_buffer = BytesIO(sample_bytes)
container = av.open(data_buffer, format="h264", mode="r")
video_stream = container.streams.video[0]
# Decode to the target frame
frame = None
for packet, time in zip(
container.demux(video_stream), sample_times, strict=False
):
packet.time_base = Fraction(1, 1_000_000_000)
packet.pts = int(time - sample_times[0])
packet.dts = packet.pts
for decoded_frame in packet.decode():
frame = decoded_frame
if isinstance(frame, av.VideoFrame):
image = np.asarray(frame.to_image())
print(
f"Efficiently decoded frame {target_frame_index} "
f"with shape: {image.shape}"
)
```
This approach is especially beneficial for:
- Long video sequences where decoding from the start is expensive
- Random access patterns where you need to jump to arbitrary frames
- High-resolution video where bandwidth and decode time are significant
- Interactive applications that need to seek to specific timestamps
## Exporting to MP4 (remuxing)
You can export video stream data to an MP4 file without re-encoding.
This is called "remuxing", the encoded samples are simply repackaged into a container format.
```python
# Query all video samples
video_df = df.select("log_time", "/video_stream:VideoStream:sample")
pa_table = pa.table(video_df)
all_times = pa_table["log_time"]
all_samples = pa_table["/video_stream:VideoStream:sample"]
# Concatenate samples into a single byte buffer
sample_bytes = np.concatenate([
sample[0] for sample in all_samples.to_numpy()
]).tobytes()
sample_bytes_io = BytesIO(sample_bytes)
# Setup input container (H.264 Annex B stream)
input_container = av.open(sample_bytes_io, mode="r", format="h264")
input_stream = input_container.streams.video[0]
# Setup output container (MP4)
output_path = TMP_DIR / "output.mp4"
output_container = av.open(output_path, mode="w")
output_stream = output_container.add_stream_from_template(input_stream)
# Remux packets with correct timestamps
start_time = all_times.chunk(0)[0]
for packet, time in zip(
input_container.demux(input_stream), all_times, strict=False
):
packet.time_base = Fraction(1, 1_000_000_000)
packet.pts = int(time.value - start_time.value)
packet.dts = packet.pts
packet.stream = output_stream
output_container.mux(packet)
input_container.close()
output_container.close()
print(f"Exported video to {output_path}")
```
## Important considerations
### Keyframe handling
Video streams often use inter-frame compression where most frames only store the difference from previous frames.
`av` handles keyframe detection internally, but for efficient random access to specific frames,
you may want to log keyframe indicators separately at recording time.
### Timestamp handling
Video timestamps in Rerun are typically stored in nanoseconds.
When using PyAV for decoding or muxing, ensure you set the correct `time_base` (typically `Fraction(1, 1_000_000_000)`).
### B-frames
Currently, Rerun's [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) does not support B-frames,
so `dts` (decode timestamp) equals `pts` (presentation timestamp).
# Overview
Rerun is the Unified Data Layer for Physical AI.
The Rerun SDK connects to a catalog server, which allows you to store, retrieve, and query over large amounts of data, and integrates with the SDK so you can browse and inspect the data visually.
The Rerun SDK includes a simplified open-source catalog server that is API compatible with Rerun Hub, our managed offering.
The open-source server loads everything into memory, which makes it fast and simple to operate for very small datasets, which in turn makes it perfect for quick testing and local experimentation.
See the [how-to guide for the open-source server](https://rerun.io/docs/howto/query-and-transform/get-data-out.md) for more details on launching and connecting to the server.
# Query images
Images are incredibly useful, however there are many ways to store and manipulate them.
This example focuses on querying image frames from a catalog server.
The dependencies in this example require `rerun-sdk[all]`.
## Setup
Simplified setup to launch the local server for demonstration.
In practice you'll connect to your cloud instance.
```python
from __future__ import annotations
from io import BytesIO
from pathlib import Path
import numpy as np
import pyarrow as pa
from datafusion import col
from PIL import Image
import rerun as rr
sample_video_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "video_sample"
)
server = rr.server.Server(datasets={"video_dataset": sample_video_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="video_dataset")
df = dataset.filter_contents([
"/compressed_images/**",
"/raw_images/**",
]).reader(index="log_time")
times = pa.table(df.select("log_time"))["log_time"].to_numpy()
```
## Compressed image
Compressed images are just stored as a string of bytes, so you can query them directly and transform back into a raw image.
```python
column_name = "/compressed_images:EncodedImage:blob"
row = df.filter(col("log_time") == times[0]).select(column_name)
image_byte_array = pa.table(row)[column_name].to_numpy()[0][0]
image = np.asarray(Image.open(BytesIO(image_byte_array.tobytes())))
print(f"{image.shape=}")
```
## Raw image
Raw images are stored in a flattened layout, so we need to reshape them.
These format details are written to the RRD when images are logged.
```python
content_column = "/raw_images:Image:buffer"
format_column = "/raw_images:Image:format"
row = df.filter(col("log_time") == times[0]).select(
content_column, format_column
)
table = pa.table(row)
format_details = table[format_column][0][0]
flattened_image = table[content_column].to_numpy()[0][0]
num_channels = rr.datatypes.color_model.ColorModel.auto(
int(format_details["color_model"].as_py())
).num_channels()
image = flattened_image.reshape(
format_details["height"].as_py(),
format_details["width"].as_py(),
num_channels,
)
print(f"{image.shape=}")
```
# Time-align data
Real-world data is usually not time-aligned.
Rerun provides capabilities to simplify time alignment.
One common use case is to fill forward to run compute at a fixed frequency.
This example demonstrates how Rerun simplifies that process.
The dependencies in this example require `rerun-sdk[all]`, and `pandas` because python datetimes only support microsecond precision.
## Setup
Simplified setup to launch the local server for demonstration.
In practice you'll connect to your cloud instance.
```python
from __future__ import annotations
from pathlib import Path
import numpy as np
from datafusion import col
import rerun as rr
sample_5_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="sample_dataset")
```
## Extract desired timepoints
Select start and end time of data, downsample to a fixed frequency, and specify those as the desired output timestamps.
```python
view = dataset.filter_segments(
"ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s"
).filter_contents("/observation/joint_positions")
ranges = view.get_index_ranges().to_arrow_table()
min_time = ranges["real_time:start"].to_numpy().flatten()
max_time = ranges["real_time:end"].to_numpy().flatten()
desired_timestamps = np.arange(
min_time[0], max_time[0], np.timedelta64(100, "ms")
) # 10Hz
```
## Time-align data
Select the timeline, columns, and episode of interest.
Extract rows at the specified time points, and fill forward to eliminate sparse entries.
Finally, filter out nulls for initial sensor state that cannot be resolved with forward fill.
```python
# Select columns of interest
# specify desired timestamps
# forward fill to specified time for alignment
fixed_hz = (
dataset
.filter_segments("ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s")
.filter_contents(["/observation/joint_positions", "/camera/ext1/**"])
.reader(
index="real_time",
using_index_values=desired_timestamps,
fill_latest_at=True,
)
)
# Filter out partially sparse rows (since one column may start before the other)
fixed_hz_filtered = fixed_hz.filter(
col("/observation/joint_positions:Scalars:scalars").is_not_null(),
col("/camera/ext1:VideoStream:sample").is_not_null(),
)
```
# Query data out of Rerun
Rerun comes with the ability to get data out of Rerun from code. This page provides an overview of the API, as well as recipes to load the data in popular packages such as [Pandas](https://pandas.pydata.org), [Polars](https://pola.rs), and [DuckDB](https://duckdb.org).
## Starting a server with recordings
The first step to query data is to start a catalog server and load it with a dataset containing your recording.
See the [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) docs for more details on how datasets are organized in Rerun.
```python
import rerun as rr
# Start a server with one or more .rrd files
with rr.server.Server(datasets={"my_dataset": ["recording.rrd"]}) as server:
client = server.client()
dataset = client.get_dataset("my_dataset")
```
The server can host multiple datasets. Each dataset maps to either a list of `.rrd` files or a directory (which will be scanned for `.rrd` files):
```python
with rr.server.Server(
datasets={
# Explicit list of RRD files
"dataset1": ["recording1.rrd", "recording2.rrd"],
# Directory containing RRD files
"dataset2": "/path/to/recordings_dir",
}
) as server:
client = server.client()
# Access each dataset by name
ds1 = client.get_dataset("dataset1")
ds2 = client.get_dataset("dataset2")
```
When multiple recordings are loaded into a dataset, each gets mapped to a separate segment whose ID is the corresponding recording ID.
You can also start a longer running server in a separate process and connect to it by its local address.
In one file or terminal launch the server and print its address,
```python
server = rr.server.Server()
print(server.url())
```
in a separate file or terminal connect to that url
```python
client = rr.catalog.CatalogClient(server.url())
```
## Adding new datasets
New datasets can also be created or appended after the server is launched:
```python
dataset = client.create_dataset(
name="oss_demo",
)
dataset.register(Path("/path/to/recording/recording.rrd").resolve().as_uri()).wait()
```
## Viewing datasets
Either specify the network location with the CLI at launch:
```console
rerun connect localhost:51234
```
or open the command palette in the viewer (`cmd/ctrl + P` or via the menu) and enter/select `Add Redap server`.
Set the scheme to `http` and enter the hostname and port in the dialog.
## Inspecting the schema
The content of a dataset can be inspected using the `schema()` method:
```python
schema = dataset.schema()
schema.index_columns() # list of all index columns (timelines)
schema.component_columns() # list of all component columns
```
## Querying a dataset using `reader`
The primary means of querying data is the `reader()` method. In its simplest form, it is used as follows:
```python
df = dataset.reader(index="frame_nr")
print(df)
```
The returned object is a [`datafusion.DataFrame`](https://datafusion.apache.org/python/autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame). Rerun's query APIs heavily rely on [DataFusion](https://datafusion.apache.org), which offers a rich set of data filtering, manipulation, and conversion tools.
When calling `reader()`, an index column must be specified. It can be any of the recording's timelines. Each row of the view will correspond to a unique value of the index column. It is also possible to query the dataset using `index=None`. In this case, only the `static=True` data will be returned.
By default, when performing a query on a dataset, data for all its segments is returned. An additional `"rerun_segment_id"` column is added to the dataframe to indicate which segment each row belongs to.
An often used parameter of the `reader()` method is `fill_latest_at=True`. When used, all `null` data will be filled with a latest-at value, similarly to how the viewer works.
## Querying a subset of a dataset
In general, datasets can be arbitrarily large, and it is often useful to query only a subset of it. This is achieved using `DatasetView` objects:
```python
# Filter by entity paths
dataset_view = dataset.filter_contents(["/world/robot/**", "/sensors/**"])
# Filter by segment IDs (recording IDs)
dataset_view = dataset.filter_segments(["recording_001", "recording_002"])
# Chain filters
dataset_view = dataset.filter_contents(["/world/**"]).filter_segments(["recording_001"])
```
`DatasetView` instances have the exact same `reader()` method as the original dataset:
```python
df = dataset_view.reader(index="frame_nr")
print(df)
```
## Filtering with DataFusion
DataFusion offers a rich set of filtering, projection, and joining capabilities. Check the [DataFusion Python documentation](https://datafusion.apache.org/python/) for details.
For illustration, here are a few simple examples:
```python
from datafusion import col
df = dataset.reader(index="frame_nr")
# Filter by index range
df = df.filter(col("frame_nr") >= 0).filter(col("frame_nr") <= 100)
# Filter by column not null
df = df.filter(col("/world/robot:Position3D:positions").is_not_null())
# Select specific columns
df = df.select("frame_nr", "/world/robot:Position3D:positions")
```
## Converting to other formats
Likewise, DataFusion offers a rich set of tools to convert a dataframe to various formats.
### Load data to a PyArrow `Table`
```python
import rerun as rr
with rr.server.Server(datasets={"my_dataset": ["recording.rrd"]}) as server:
dataset = server.client().get_dataset("my_dataset")
table = dataset.reader(index="frame_nr").to_arrow_table()
```
### Load data to a Pandas dataframe
```python
import rerun as rr
with rr.server.Server(datasets={"my_dataset": ["recording.rrd"]}) as server:
dataset = server.client().get_dataset("my_dataset")
df = dataset.reader(index="frame_nr").to_pandas()
```
### Load data to a Polars dataframe
```python
import rerun as rr
import polars as pl
with rr.server.Server(datasets={"my_dataset": ["recording.rrd"]}) as server:
dataset = server.client().get_dataset("my_dataset")
df = pl.from_arrow(dataset.reader(index="frame_nr").to_arrow_table())
```
### Load data to a DuckDB relation
```python
import rerun as rr
import duckdb
with rr.server.Server(datasets={"my_dataset": ["recording.rrd"]}) as server:
dataset = server.client().get_dataset("my_dataset")
table = dataset.reader(index="frame_nr").to_arrow_table()
rel = duckdb.arrow(table)
```
# View Operations
Robotics data has many sensors and many columns.
In order to more narrowly specify relevant content for further dataframe operations you first generate a view.
This view can filter on episode, time, column name etc.
This example shows specific instances highlighting these capabilities.
The dependencies in this example are contained in `rerun-sdk[all]`.
## Setup
Simplified setup.
Perform imports and launch local server for demonstration.
Extract an initial view `observations` that we later refine before generating a dataframe.
```python
from __future__ import annotations
from pathlib import Path
import pyarrow as pa
from datafusion import col
import rerun as rr
sample_5_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="sample_dataset")
observations = dataset.filter_contents("/observation/**")
```
## Filtering on episode and time
Limit the scope of the view so that only a subset of data will ever be considered client side.
Pick a specific episode by id, and a time range.
```python
episode = "ILIAD_50aee79f_2023_07_12_20h_55m_08s"
start = 1689220508
end = start + 5
filtered_view = dataset.filter_segments(episode).filter_contents(
"/observation/**"
)
filtered_df = filtered_view.reader(index="real_time")
filtered_df = filtered_df.filter(
(col("real_time") >= start) & (col("real_time") < end)
)
```
## Querying static data
So far we've been selecting `real_time` as the index. However, some data might not change in time (e.g. static transformations)
and thus isn't aligned with a time index. We specify this static timeline with `None`.
```python
instructions = dataset.filter_contents("/language_instruction/**").reader(
index=None
)
# Sort to ensure documented output is always correct
instructions = instructions.sort("/language_instruction:TextDocument:text")
instructions_tbl = pa.table(instructions)
instructions_tbl["/language_instruction:TextDocument:text"][0]
```
# Query Performance Tuning
This is a loose collection of considerations when querying Rerun datasets.
Over time baseline performance will improve, rendering some of these approaches unnecessary.
Since Rerun depends on [DataFusion](https://datafusion.apache.org/), some of these approaches are observations from our own usage.
First, generate a DataFrame for comparison:
```python
sample_video_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "video_sample"
)
server = rr.server.Server(datasets={"video_dataset": sample_video_path})
# Using OSS server for demonstration but in practice replace with
# the URL of your cloud instance
CATALOG_URL = server.url()
client = rr.catalog.CatalogClient(CATALOG_URL)
dataset = client.get_dataset(name="video_dataset")
df = dataset.filter_contents([
"/compressed_images/**",
"/raw_images/**",
]).reader(index="log_time")
```
## Extract Python types from a DataFrame
DataFusion is a streaming query engine, which allows for processing arbitrarily large amounts of data.
When working with smaller or filtered-down datasets that fit into memory, you can extract data into Python variables for further post processing.
In these examples, we convert DataFrames to [PyArrow](https://arrow.apache.org/docs/python/index.html) tables to materialize them in memory.
Similar patterns using Polars or Pandas also apply.
### Prefer to_numpy
This is technically a [PyArrow](https://arrow.apache.org/docs/python/index.html) and general Python detail.
For example, when extracting data from a PyArrow table, `to_pylist` can be multiple orders of magnitude slower, even when using `to_numpy(zero_copy_only=False)`.
```python
table = pa.table(df)
table["log_time"].to_numpy()
# vs.
table["log_time"].to_pylist()
```
## Fine-tune data collection
Similar to the approach described above to collect a DataFusion `DataFrame` into a PyArrow table, you can instead collect the results in memory and keep them as a `DataFrame`.
Then any operations on this in-memory (cached) `DataFrame` are typically _very_ fast.
```python
df.count() # has to pull some data
df.count() # has to pull same data again
# vs.
cache_df = df.cache() # materializes table in memory
cache_df.count() # basically free
cache_df.count() # basically free
```
## Leverage sparsity to minimize scans
In a write once, read many paradigm adding an additional sparse column can enable cheap access to data of interest via filtering.
The catalog server has the ability to "push down" filters to greatly reduce the amount of data returned, improving query performance.
In this example we take advantage of this fact by filtering based on a sparse marker we have intentionally inserted into the recording.
```python
# Create a new sparse layer identifying interesting events
segment_id = dataset.segment_ids()[0]
second_to_last_timestamp = pa.table(df)["log_time"].to_numpy()[-2]
with rr.RecordingStream("rerun_example_layer", recording_id=segment_id) as rec:
rec.save(RRD_PATH)
rec.set_time("log_time", timestamp=second_to_last_timestamp)
rec.log("/events", rr.AnyValues(flag=True))
dataset.register([Path(RRD_PATH).as_uri()], layer_name="event_layer")
# Read dataframe including new sparse layer
df_with_flag = dataset.filter_contents([
"/compressed_images/**",
"/raw_images/**",
"/events/**",
]).reader(index="log_time")
# This filter only looks at the single row in events
df_with_flag.filter(col("/events:flag").is_not_null())
# vs. using row_number which requires scanning all rows
df_with_row_number = df.with_column(
"row_num",
F.row_number(order_by="log_time"),
)
df_with_row_number.filter(col("row_num") == df_with_row_number.count() - 1)
```
# Generate segment URLs in dataframes
The [`segment_url`](https://ref.rerun.io/docs/python/stable/common/utilities/#rerun.utilities.datafusion.functions.url_generation.segment_url) DataFusion utility can be used to generate Rerun URLs that are clickable within the viewer.
The generated URLs can optionally seek to a timestamp, select a time range, or select an entity path.
## Setup
We start by loading sample data in a local catalog server instance and creating a table with some segment metadata.
```python
from __future__ import annotations
from datetime import datetime, timedelta
from pathlib import Path
import pyarrow as pa
from datafusion import lit
import rerun as rr
from rerun.utilities.datafusion.functions.url_generation import segment_url
sample_5_path = (
Path(__file__).parents[5] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
client = server.client()
dataset = client.get_dataset(name="sample_dataset")
# Pick 3 deterministic segment IDs and create a view filtered to them
segment_ids = sorted(dataset.segment_ids())[:3]
view = dataset.filter_segments(segment_ids)
# Build a synthetic metadata table keyed by rerun_segment_id
base_time = datetime(2023, 11, 14, 22, 13, 20)
event_times = [base_time + timedelta(seconds=i) for i in range(3)]
meta = pa.record_batch(
{
"rerun_segment_id": segment_ids,
"event_time": pa.array(event_times, type=pa.timestamp("ns")),
"range_start": pa.array(event_times, type=pa.timestamp("ns")),
"range_end": pa.array(
[t + timedelta(milliseconds=500) for t in event_times],
type=pa.timestamp("ns"),
),
"entity_path": [
"/camera/rgb",
"/observation/joint_positions",
"/observation/gripper_state",
],
},
)
ctx = client.ctx
meta_df = ctx.from_arrow(meta)
```
## Basic URL
With no extra arguments, `segment_url` produces a URL that opens the segment in the viewer.
```python
basic = view.segment_table().select("rerun_segment_id").sort("rerun_segment_id")
basic = basic.with_column("url", segment_url(dataset))
for url in basic.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=
rerun+http://localhost:51234/dataset/?segment_id=
rerun+http://localhost:51234/dataset/?segment_id=
```
## Specify the time cursor position
Pass `timestamp` and `timeline_name` to generate a URL that tells the viewer to activate a specific timeline and set
the time cursor to a specific value.
If `timestamp` is a string, it will be interpreted as a column name.
Alternatively, any DataFusion expression can be provided, including a literal.
```python
ts = view.segment_table(join_meta=meta_df).select(
"rerun_segment_id", "event_time"
)
ts = ts.sort("rerun_segment_id")
ts = ts.with_column(
"url",
segment_url(dataset, timestamp="event_time", timeline_name="real_time"),
)
for url in ts.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=#when=real_time@2023-11-14T22:13:20Z
rerun+http://localhost:51234/dataset/?segment_id=#when=real_time@2023-11-14T22:13:21Z
rerun+http://localhost:51234/dataset/?segment_id=#when=real_time@2023-11-14T22:13:22Z
```
## Selecting a time range
Pass `time_range_start` and `time_range_end` together with `timeline_name` to generate a URL that specifies a time range to be selected.
Both can be a column name or a DataFusion expression.
```python
tr = view.segment_table(join_meta=meta_df).select(
"rerun_segment_id", "range_start", "range_end"
)
tr = tr.sort("rerun_segment_id")
tr = tr.with_column(
"url",
segment_url(
dataset,
time_range_start="range_start",
time_range_end="range_end",
timeline_name="real_time",
),
)
for url in tr.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=#time_selection=real_time@2023-11-14T22:13:20Z..2023-11-14T22:13:20.5Z
rerun+http://localhost:51234/dataset/?segment_id=#time_selection=real_time@2023-11-14T22:13:21Z..2023-11-14T22:13:21.5Z
rerun+http://localhost:51234/dataset/?segment_id=#time_selection=real_time@2023-11-14T22:13:22Z..2023-11-14T22:13:22.5Z
```
## Selecting an entity
Pass `selection` to generate a URL that specifies which entity path, instance, and/or component to select.
The value must be a string using entity path syntax, optionally followed by an instance index in brackets
and/or a component name after a colon.
For example: `/world/points`, `/world/points[#42]`, `/world/points:Color`, or `/world/points[#42]:Color`.
```python
sel = view.segment_table(join_meta=meta_df).select(
"rerun_segment_id", "entity_path"
)
sel = sel.sort("rerun_segment_id")
sel = sel.with_column("url", segment_url(dataset, selection="entity_path"))
for url in sel.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=#selection=/camera/rgb
rerun+http://localhost:51234/dataset/?segment_id=#selection=/observation/joint_positions
rerun+http://localhost:51234/dataset/?segment_id=#selection=/observation/gripper_state
```
## Combining features
All three features can be used together. The generated URL includes every fragment that was specified.
```python
combined = view.segment_table(join_meta=meta_df).select(
"rerun_segment_id", "event_time", "range_start", "range_end", "entity_path"
)
combined = combined.sort("rerun_segment_id")
combined = combined.with_column(
"url",
segment_url(
dataset,
timestamp="event_time",
timeline_name="real_time",
time_range_start="range_start",
time_range_end="range_end",
selection="entity_path",
),
)
for url in combined.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=#selection=/camera/rgb&when=real_time@2023-11-14T22:13:20Z&time_selection=real_time@2023-11-14T22:13:20Z..2023-11-14T22:13:20.5Z
rerun+http://localhost:51234/dataset/?segment_id=#selection=/observation/joint_positions&when=real_time@2023-11-14T22:13:21Z&time_selection=real_time@2023-11-14T22:13:21Z..2023-11-14T22:13:21.5Z
rerun+http://localhost:51234/dataset/?segment_id=#selection=/observation/gripper_state&when=real_time@2023-11-14T22:13:22Z&time_selection=real_time@2023-11-14T22:13:22Z..2023-11-14T22:13:22.5Z
```
## Using expressions
Every parameter that accepts a column name string also accepts an arbitrary DataFusion expression.
This is useful when you want to supply a constant value for all rows using `lit()` or build more advanced expressions.
```python
expr = view.segment_table(join_meta=meta_df).select(
"rerun_segment_id", "event_time"
)
expr = expr.sort("rerun_segment_id")
expr = expr.with_column(
"url",
segment_url(
dataset,
timestamp="event_time",
timeline_name="real_time",
selection=lit("/camera/rgb"),
),
)
for url in expr.select("url").to_pydict()["url"]:
print(url)
```
Output:
```
rerun+http://localhost:51234/dataset/?segment_id=#selection=/camera/rgb&when=real_time@2023-11-14T22:13:20Z
rerun+http://localhost:51234/dataset/?segment_id=#selection=/camera/rgb&when=real_time@2023-11-14T22:13:21Z
rerun+http://localhost:51234/dataset/?segment_id=#selection=/camera/rgb&when=real_time@2023-11-14T22:13:22Z
```
# Embed Rerun in notebooks
Starting with version 0.15.1, Rerun has improved support for embedding the Rerun Viewer directly within IPython-style
notebooks. This makes it easy to iterate on API calls as well as to share data with others.
Rerun has been tested with:
- [Jupyter Notebook Classic](https://jupyter.org/)
- [Jupyter Lab](https://jupyter.org/)
- [VSCode](https://code.visualstudio.com/blogs/2021/08/05/notebooks)
- [Google Colab](https://colab.research.google.com/)
To begin, install the `rerun-sdk` package with the `notebook` extra:
```sh
pip install rerun-sdk[notebook]
```
This installs both [rerun-sdk](https://pypi.org/project/rerun-sdk/) and [rerun-notebook](https://pypi.org/project/rerun-notebook/).
## The APIs
When using the Rerun logging APIs, by default, the logged messages are buffered in-memory until
you send them to a sink such as via `rr.connect_grpc()` or `rr.save()`.
When using Rerun in a notebook, rather than using the other sinks, you have the option to use [`rr.notebook_show()`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.notebook_show). This method embeds the [web viewer](https://rerun.io/docs/howto/integrations/embed-web.md) using the IPython `display` mechanism in the cell output, and sends the current recording data to it.
Once the viewer is open, any subsequent `rr.log()` calls will send their data directly to the viewer,
without any intermediate buffering.
For example:
```python
import rerun as rr
from numpy.random import default_rng
rr.init("rerun_example_notebook")
rng = default_rng(12345)
positions = rng.uniform(-5, 5, size=[10, 3])
colors = rng.uniform(0, 255, size=[10, 3])
radii = rng.uniform(0, 1, size=[10])
rr.log("random", rr.Points3D(positions, colors=colors, radii=radii))
rr.notebook_show()
```
This is similar to calling `rr.connect_grpc()` or `rr.serve()` in that it configures the Rerun SDK to send data to a viewer instance.
Note that the call to `rr.notebook_show()` drains the recording of its data. This means that any subsequent calls to `rr.notebook_show()`
will not result in the same data being displayed, because it has already been removed from the recording.
Support for this is tracked in [#6612](https://github.com/rerun-io/rerun/issues/6612).
If you wish to start a new recording, you can call `rr.init()` again.
The `notebook_show()` method also takes optional arguments for specifying the width and height of the viewer. For example:
```python
rr.notebook_show(width=400, height=400)
```
## Reacting to events in the Viewer
It is possible to register a callback to be triggered when certain Viewer events happen.
For example, here is how you can track which entities are currently selected in the Viewer:
```python
from rerun.notebook import Viewer, ViewerEvent
selected_entities = []
def on_event(event: ViewerEvent):
global selected_entities
selected_entities = [] # clear the list
if event.type == "selection_change":
for item in event.items:
if item.type == "entity":
selected_entities.append(item.entity_path)
viewer = Viewer()
viewer.on_event(on_event)
display(viewer)
```
Whenever an entity is selected in the Viewer, `selected_entities.value` changes. The payload includes other useful information,
such as the position of the selection within a 2D or 3D view.
For a more complete example, see [callbacks.ipynb](https://github.com/rerun-io/rerun/blob/main/examples/python/notebook_callbacks/notebook_callbacks.ipynb).
## Working with blueprints
[Blueprints](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md) can also be used with `notebook_show()` by providing a `blueprint`
parameter.
For example
```python
blueprint = rrb.Blueprint(
rrb.Horizontal(rrb.Spatial3DView(origin="/world"), rrb.Spatial2DView(origin="/world/camera"), column_shares=[2, 1]),
)
rr.notebook_show(blueprint=blueprint)
```
Because blueprint types implement `_ipython_display_`, you can also just end any cell with a blueprint
object, and it will call `notebook_show()` behind the scenes.
```python
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_image")
rng = np.random.default_rng(12345)
image1 = rng.uniform(0, 255, size=[24, 64, 3])
image2 = rng.uniform(0, 255, size=[24, 64, 1])
rr.log("image1", rr.Image(image1))
rr.log("image2", rr.Image(image2))
rrb.Vertical(rrb.Spatial2DView(origin="/image1"), rrb.Spatial2DView(origin="/image2"))
```
## Streaming data
The notebook integration supports streaming data to the viewer during cell execution.
You can call `rr.notebook_show()` at any point after calling `rr.init()`, and any
`rr.log()` calls will be sent to the viewer in real-time.
```python
import math
from time import sleep
import numpy as np
import rerun as rr
from rerun.utilities import build_color_grid
rr.init("rerun_example_notebook")
rr.notebook_show()
STEPS = 100
twists = math.pi * np.sin(np.linspace(0, math.tau, STEPS)) / 4
for t in range(STEPS):
sleep(0.05) # delay to simulate a long-running computation
rr.set_time("step", sequence=t)
cube = build_color_grid(10, 10, 10, twist=twists[t])
rr.log("cube", rr.Points3D(cube.positions, colors=cube.colors, radii=0.5))
```
## Some working examples
To experiment with notebooks yourself, there are a few options.
### Running locally
The GitHub repo includes a [notebook example](https://github.com/rerun-io/rerun/blob/main/examples/python/notebook/cube.ipynb).
If you have a local checkout of Rerun, you can:
```bash
$ cd examples/python/notebook
$ pip install -r requirements.txt
$ jupyter notebook cube.ipynb
```
This will open a browser window showing the notebook where you can follow along.
### Running in Google Colab
We also host a copy of the notebook in [Google Colab](https://colab.research.google.com/drive/1R9I7s4o6wydQC_zkybqaSRFTtlEaked_)
Note that if you copy and run the notebook yourself, the first Cell installs Rerun into the Colab environment.
After running this cell you will need to restart the Runtime for the Rerun package to show up successfully.
## Limitations
Browsers have limitations in the amount of memory usable by a single tab. If you are working with large datasets,
you may run into browser tab crashes due to out-of-memory errors.
If you encounter the issue, you can try to use the `save()` API to save the data to a file and share it as a standalone asset.
## Future work
We are actively working on improving the notebook experience and welcome any [feedback or suggestions](https://rerun.io/feedback).
The ongoing roadmap is being tracked in [GitHub issue #1815](https://github.com/rerun-io/rerun/issues/1815).
# Use Rerun with ROS 2
Rerun does not yet have native ROS support, but many of the concepts in ROS and Rerun
line up fairly well. In this guide, you will learn how to write a simple ROS 2 Python node
that subscribes to some common ROS topics and logs them to Rerun.
For information on future plans to enable more native ROS support
see [#1537](https://github.com/rerun-io/rerun/issues/1537).
In case you have recorded data, you may also want to read our documentation on [using MCAP](https://rerun.io/docs/howto/logging-and-ingestion/mcap.md).
The following is primarily intended for existing ROS 2 users. It will not spend much time
covering how to use ROS 2 itself. If you are a Rerun user that is curious about ROS,
please consult the [ROS 2 Documentation](https://docs.ros.org) instead.
All of the code for this guide can be found on GitHub in
[rerun/examples/python/ros_node](https://github.com/rerun-io/rerun/blob/main/examples/python/ros_node/).
---
Other relevant tutorials:
- [Log and Ingest Tutorial](https://rerun.io/docs/getting-started/data-in.md)
- [Viewer Walkthrough](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md)
- [Transforms & Coordinate Frames](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md)
- [Loading URDF models](https://rerun.io/docs/howto/logging-and-ingestion/urdf.md)
- [Working with MCAP](https://rerun.io/docs/howto/logging-and-ingestion/mcap.md)
If you're new to Rerun or wonder about differences to RViz, we recommend also to read the *"What is Rerun for?"* introduction in our README (see [here](https://github.com/rerun-io/rerun?tab=readme-ov-file#what-is-rerun-for)).
## Install and run the example
All steps that are required to install and run this example are explained in the example's description, which can be found [here](https://rerun.io/examples/robotics/ros_node) or in the [README.md](https://github.com/rerun-io/rerun/tree/main/examples/python/ros_node) of the example's source code.
## Code explanation
It may be helpful to open [rerun/examples/python/ros_node/main.py](https://github.com/rerun-io/rerun/blob/latest/examples/python/ros_node/main.py)
to follow along.
At a very high level, for each ROS message we are interested in, we create a
subscriber with a callback that does some form of data conversion and then logs the data to Rerun.
In most cases, this conversion is either trivial or easy to do with utilities from the ROS ecosystem.
For simplicity, this example uses the rosclpy `MultiThreadedExecutor` and `ReentrantCallbackGroup` for each topic. This
allows each callback thread to do TF lookups without blocking the other incoming messages. More advanced ROS execution
models and using asynchronous TF lookups are outside the scope of this guide.
### Updating time
First of all, we want our messages to show up on the timeline based on their _stamped_ time rather than the
time that they were received by the listener, or relayed to Rerun.
To do this, we will use a Rerun timeline called `ros_time`.
Each callback follows a common pattern of updating `ros_time` based on the stamped time of the message that was
received.
```python
def some_msg_callback(self, msg: Msg):
time = Time.from_msg(msg.header.stamp)
rr.set_time("ros_time", timestamp=np.datetime64(time.nanoseconds, "ns"))
```
This timestamp will apply to all subsequent log calls on in this callback (on this thread) until the time is updated
again.
### TF to rr.Transform3D
Next, we need to map the [ROS TF2](https://docs.ros.org/en/humble/Concepts/About-Tf2.html) transforms to the corresponding Rerun archetype.
Since Rerun 0.28, the [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md) archetype supports parent/child frame relationships, which makes our conversion step straight-forward. We just have to remember that the ROS [TFMessage](https://docs.ros2.org/foxy/api/tf2_msgs/msg/TFMessage.html) is a container for multiple transforms that have individual timestamps each and set the time accordingly.
By specifying the parent and child frames, we can log all transforms to the same entity path, similar to the TF topic in ROS.
To make sense of these transforms in the rest of our logged data, we also have to associate them to their respective frame name using [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)s.
See the laser scan section below for an example.
More information about the different ways Rerun can handle transforms can be found [here](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md).
```python
def tf_callback(self, tf_msg: TFMessage) -> None:
for transform in tf_msg.transforms:
time = Time.from_msg(transform.header.stamp)
rr.set_time("ros_time", timestamp=np.datetime64(time.nanoseconds, "ns"))
rr.log(
"transforms",
rr.Transform3D(
translation=[
transform.transform.translation.x,
transform.transform.translation.y,
transform.transform.translation.z,
],
rotation=rr.Quaternion(
xyzw=[
transform.transform.rotation.x,
transform.transform.rotation.y,
transform.transform.rotation.z,
transform.transform.rotation.w,
]
),
parent_frame=transform.header.frame_id,
child_frame=transform.child_frame_id,
),
)
```
### `robot_description` (URDF)
Rerun features a built-in importer for URDF, so we can just forward the string received on the `/robot_description` topic to it.
More information about how to use URDF with Rerun can be found [here](https://rerun.io/docs/howto/logging-and-ingestion/urdf.md).
```python
def urdf_callback(self, urdf_msg: String) -> None:
# NOTE: file_path is not known here, robot.urdf is just a placeholder to let
# Rerun know the file type. Since we run this example in a ROS environment,
# Rerun can use AMENT_PREFIX_PATH etc to resolve asset paths of the URDF.
rr.log_file_from_contents(
file_path="robot.urdf",
file_contents=urdf_msg.data.encode("utf-8"),
entity_path_prefix="urdf",
static=True,
)
```
### LaserScan to rr.LineStrips3D
Rerun does not yet have native support for a `LaserScan` style primitive, so we need
to do a bit of additional transformation logic (see: [#1534](https://github.com/rerun-io/rerun/issues/1534).)
First, we convert the scan into a point-cloud using the `laser_geometry` package.
We could have logged the Points directly using `rr.Points3D`, but for
the sake of this demo, we wanted to instead log a laser scan as a bunch of lines
in a similar fashion to how it is depicted in gazebo.
We generate a second matching set of points for each ray projected out 0.3m from
the origin and then interlace the two sets of points using Numpy hstack and reshape.
This results in a set of alternating points defining rays from the origin to each
laser scan result, which is the format expected by `rr.LineStrips3D`.
By logging also scan's `frame_id` as a [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md), we make sure that Rerun visualizes the lines at the right location in the transform hierarchy.
```python
def __init__(self) -> None:
# β¦
self.laser_proj = laser_geometry.laser_geometry.LaserProjection()
def scan_callback(self, scan: LaserScan) -> None:
time = Time.from_msg(scan.header.stamp)
rr.set_time("ros_time", timestamp=np.datetime64(time.nanoseconds, "ns"))
# Project the laser scan to a collection of points
points = self.laser_proj.projectLaser(scan)
pts = point_cloud2.read_points(points, field_names=["x", "y", "z"], skip_nans=True)
pts = structured_to_unstructured(pts)
# Turn every pt into a line-segment from the origin to the point.
origin = (pts / np.linalg.norm(pts, axis=1).reshape(-1, 1)) * 0.3
segs = np.hstack([origin, pts]).reshape(pts.shape[0] * 2, 3)
rr.log("scan", rr.LineStrips3D(segs, radii=0.0025, colors=[255, 165, 0]))
rr.log("scan", rr.CoordinateFrame(frame=scan.header.frame_id))
```
### OccupancyGrid to rr.GridMap
ROS [`nav_msgs/OccupancyGrid`](https://docs.ros2.org/latest/api/nav_msgs/msg/OccupancyGrid.html) messages map directly to Rerun's [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md) archetype.
This example subscribes to the static map and the local & global costmap topics, logging them with Rerun's RViz-compatible `RvizMap` and `RvizCostmap` colormaps and with draw-order values for defined layering.
Most fields are a 1:1 mapping: the occupancy data becomes the `GridMap` image data, `info.resolution` becomes the cell size, and `info.origin` defines the map pose.
The main caveat is row order: ROS occupancy grids start at the map's bottom-left cell, while regular image buffers as used by Rerun's `GridMap` are top-row first.
The example therefore flips the rows before logging the grid data.
### Camera info and images
ROS Images can also be mapped to Rerun very easily, using the `cv_bridge` package.
The output of `cv_bridge.imgmsg_to_cv2` can be fed directly into `rr.Image`.
For the camera info topic, we can use the `image_geometry` package that has a `PinholeCameraModel` that exposes the intrinsic matrix in the same structure as used by Rerun `rr.Pinhole`.
Like for the laser scan, we also have to associate the data with the correct coordinate frame.
In order to have a nice projection of the image in the pinhole frustum in Rerun's 3D view, we have to establish a relationship between the 3D extrinsic camera frame and the 2D image plane.
The first is just the `frame_id` that we get from the ROS message, while the latter is something that isn't a concept in ROS.
To distinguish the two, we just use an `_image_plane` suffix in the image plane frame name and make sure that both the pinhole and image logging use it.
```python
def __init__(self) -> None:
# β¦
self.cv_bridge = cv_bridge.CvBridge()
def cam_info_callback(self, info: CameraInfo) -> None:
"""
Logs CameraInfo as a Rerun Pinhole.
"""
time = Time.from_msg(info.header.stamp)
self.pinhole_model.from_camera_info(info)
rr.set_time("ros_time", timestamp=np.datetime64(time.nanoseconds, "ns"))
rr.log(
"rgbd_camera/camera_info",
rr.Pinhole(
resolution=[info.width, info.height],
image_from_camera=self.pinhole_model.intrinsic_matrix(),
image_plane_distance=1.0,
parent_frame=info.header.frame_id,
# Specifying a `child_frame` for the 2D image plane allows Rerun to
# visualize the pinhole frustum together with the image in 3D views.
# This has to match the coordinate frames used when logging images,
# see `image_callback` below.
child_frame=info.header.frame_id + "_image_plane",
),
)
def image_callback(self, img: Image) -> None:
time = Time.from_msg(img.header.stamp)
rr.set_time("ros_time", timestamp=np.datetime64(time.nanoseconds, "ns"))
rr.log("rgbd_camera/image", rr.Image(self.cv_bridge.imgmsg_to_cv2(img)))
# Make sure the image plane frame matches what we set in `cam_info_callback`.
rr.log("rgbd_camera/image", rr.CoordinateFrame(frame=img.header.frame_id + "_image_plane"))
```
### Others
The example also logs more data, like depth images and parts of the odometry data.
Please refer to the [source code](https://github.com/rerun-io/rerun/blob/main/examples/python/ros_node/) of the example to see the details of those.
## In summary
Although there is a non-trivial amount of code, none of it is overly complicated. Each message callback
operates independently of the others, processing an incoming message, adapting it to Rerun and then
logging it again.
There are several places where Rerun is currently missing support for primitives that will further
simplify this implementation. We will continue to update this guide as new functionality becomes
available.
While this guide has only covered a small fraction of the possible ROS messages that could
be sent to Rerun, hopefully, it has given you some tools to apply to your project.
If you find that specific functionality is lacking for your use case, please provide more
context in the existing issues or [open an new one](https://github.com/rerun-io/rerun/issues/new/choose) on GitHub.
# Integrate Rerun with native loggers
The Rerun SDK implements the native logging interfaces of its supported host languages, allowing you to transparently stream text logs logged with the native APIs into the Rerun Viewer.
The details of how to achieve that vary language by language, see the snippets below.
```python
"""Shows integration of Rerun's `TextLog` with the native logging interface."""
import logging
import rerun as rr
rr.init("rerun_example_text_log_integration", spawn=True)
# Log a text entry directly
rr.log(
"logs",
rr.TextLog("this entry has loglevel TRACE", level=rr.TextLogLevel.TRACE),
)
# Or log via a logging handler
logging.getLogger().addHandler(rr.LoggingHandler("logs/handler"))
logging.getLogger().setLevel(-1)
logging.info("This INFO log got added through the standard logging interface")
```
# Embed Rerun in Web pages
Integrating the Rerun Viewer into your web application can be accomplished either by [utilizing an iframe](#embedding-apprerunio-using-an-iframe) or by using our [JavaScript package](#using-the-javascript-package).
## Embedding `app.rerun.io` using an `
```
To implement this, fill in the placeholders:
- `RRD_URL` - The URL of the recording to display in the viewer.
- `RERUN_VERSION` - The version of the Rerun SDK used to generate the recording.
The `RRD_URL` can be a file served over `http` (e.g. `https://app.rerun.io/version/0.20.3/examples/arkit_scenes.rrd`), or a connection to an SDK using our [serve](https://www.rerun.io/docs/reference/sdk/operating-modes#serve) API (e.g. `rerun+http://localhost:4321/proxy`).
For instance:
```html
```
### Matching the host page's theme
By default, the embedded viewer follows the user's OS theme (`prefers-color-scheme`). If your host page has its own theme toggle, you can pin the viewer to match by passing `theme=dark`, `theme=light`, or `theme=system`:
```html
```
This is useful for sites whose theme can differ from the OS preference β without it, a user on a light-mode OS visiting your dark-mode page would see a bright viewer panel against a dark background.
## Using the JavaScript package
We offer JavaScript bindings to the Rerun Viewer via NPM. This method provides control over the Viewer but requires a JavaScript web application setup with a bundler.
Various packages are available:
- [@rerun-io/web-viewer](https://www.npmjs.com/package/@rerun-io/web-viewer): Suitable for JS apps without a framework or frameworks without dedicated packages.
- [@rerun-io/web-viewer-react](https://www.npmjs.com/package/@rerun-io/web-viewer-react): Designed specifically for React apps.
> [!NOTE]
> The stability of the `rrd` format is still evolving, so the package version corresponds to the supported Rerun SDK version. Therefore, `@rerun-io/web-viewer@0.10.0` can only connect to a data source (`.rrd` file, gRPC connection, etc.) originating from a Rerun SDK with version `0.10.0`!
### Basic example
To begin, install the package ([@rerun-io/web-viewer](https://www.npmjs.com/package/@rerun-io/web-viewer)) from NPM:
```
npm i @rerun-io/web-viewer
```
> [!NOTE]
> This package is compatible only with recent browser versions. If your target browser lacks support for Wasm imports or top-level await, additional plugins may be required for your bundler setup. For instance, if you're using [Vite](https://vitejs.dev/), you'll need to install [vite-plugin-wasm](https://www.npmjs.com/package/vite-plugin-wasm) and [vite-plugin-top-level-await](https://www.npmjs.com/package/vite-plugin-top-level-await) and integrate them into your `vite.config.js`.
Once installed and configured, import and use it within your application:
```js
import { WebViewer } from "@rerun-io/web-viewer";
const rrdUrl = null;
const parentElement = document.body;
const viewer = new WebViewer();
await viewer.start(rrdUrl, parentElement);
```
The Viewer creates a `` on the provided `parentElement` and executes within it.
The first argument for `start` determines the recordings to open in the viewer. It can be:
- `null` for an initially empty viewer
- a URL string to open a single recording
- an array of strings to open multiple recordings
Each URL can be either a file served over `http` or a connection to an SDK using our [serve](https://www.rerun.io/docs/reference/sdk/operating-modes#serve) API. See [web-viewer-serve-example](https://github.com/rerun-io/web-viewer-serve-example) for a full example of how to log data from our Python SDK to an embedded Rerun Viewer.
### Controlling the canvas
By default, the web viewer attempts to expand the canvas to occupy all available space. You can customize its dimensions by placing it within a container:
```html,id=embed-web-viewer-canvas-control-html
```
```css,id=embed-web-viewer-canvas-control-css
#viewer-container {
position: relative;
height: 640px;
width: 100%;
}
```
```js,id=embed-web-viewer-canvas-control-js
const parentElement = document.getElementById("viewer-container");
const viewer = new WebViewer();
await viewer.start(null, parentElement);
```
### Viewer API
The Viewer API supports adding and removing recordings:
```js,id=embed-web-viewer-api-js-open-close
const rrdUrl = "https://app.rerun.io/version/0.20.3/examples/arkit_scenes.rrd";
// Open a recording:
viewer.open(rrdUrl);
// Later onβ¦
viewer.close(rrdUrl);
```
Once finished with the Viewer, you can stop it and release all associated resources:
```js,id=embed-web-viewer-api-js-stop
viewer.stop();
```
This action also removes the canvas from the page.
You can `start` and `stop` the same `WebViewer` instance multiple times.
### Callbacks
The Viewer API also allows registering callbacks for certain events.
For example, here is how you would react to entities being selected in the Viewer:
```js
viewer.on("selection_change", (event) => {
for (const item of event.items) {
if (item.type === "entity") {
console.log(item.entity_path);
}
}
});
```
# Send entire columns at once
The [`log` API](https://rerun.io/docs/getting-started/data-in.md) is designed to extract data from your running code as it's being generated. It is, by nature, *row-oriented*.
If you already have data stored in something more *column-oriented*, it can be both a lot easier and more efficient to send it to Rerun in that form directly.
This is what the `send_columns` API is for: it lets you efficiently update the state of an entity over time, sending data for multiple index and component columns in a single operation.
> [!WARNING]
> `send_columns` API bypasses the time context and [micro-batcher](https://rerun.io/docs/reference/sdk/micro-batching.md).
>
> In contrast to the `log` API, `send_columns` does NOT add any other timelines to the data. Neither the built-in timelines `log_time` and `log_tick`, nor any [user timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md). Only the timelines explicitly included in the call to `send_columns` will be included.
To learn more about the concepts behind the columnar APIs, and the Rerun data model in general, [refer to this page](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md).
## Reference
* [π C++](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#ad17571d51185ce2fc2fc2f5c3070ad65)
* [π Python](https://ref.rerun.io/docs/python/stable/common/columnar_api/#rerun.send_columns)
* [π¦ Rust](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.send_columns)
## Examples
### Updating a scalar over time, in a single operation
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update a scalar over time.
See also the `scalar_column_updates` example, which achieves the same
thing in a single operation.
"""
from __future__ import annotations
import math
import rerun as rr
rr.init("rerun_example_scalar_row_updates", spawn=True)
for step in range(64):
rr.set_time("step", sequence=step)
rr.log("scalars", rr.Scalars(math.sin(step / 10.0)))
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update a scalar over time, in a single operation.
This is semantically equivalent to the `scalar_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_scalar_column_updates", spawn=True)
times = np.arange(0, 64)
scalars = np.sin(times / 10.0)
rr.send_columns(
"scalars",
indexes=[rr.TimeColumn("step", sequence=times)],
columns=rr.Scalars.columns(scalars=scalars),
)
```
### Updating a point cloud over time, in a single operation
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update a point cloud over time.
See also the `points3d_column_updates` example, which achieves the same
thing in a single operation.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_points3d_row_updates", spawn=True)
# Prepare a point cloud that evolves over 5 timesteps, changing the
# number of points in the process.
times = np.arange(10, 15, 1.0)
# fmt: off
positions = [
[[1.0, 0.0, 1.0], [0.5, 0.5, 2.0]],
[[1.5, -0.5, 1.5], [1.0, 1.0, 2.5], [-0.5, 1.5, 1.0], [-1.5, 0.0, 2.0]],
[[2.0, 0.0, 2.0], [1.5, -1.5, 3.0], [0.0, -2.0, 2.5], [1.0, -1.0, 3.5]],
[[-2.0, 0.0, 2.0], [-1.5, 1.5, 3.0], [-1.0, 1.0, 3.5]],
[[1.0, -1.0, 1.0], [2.0, -2.0, 2.0], [3.0, -1.0, 3.0], [2.0, 0.0, 4.0]],
]
# fmt: on
# At each timestep, all points in the cloud share the same but changing
# color and radius.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
radii = [0.05, 0.01, 0.2, 0.1, 0.3]
for i in range(5):
rr.set_time("time", duration=10 + i)
rr.log(
"points", rr.Points3D(positions[i], colors=colors[i], radii=radii[i])
)
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update a point cloud over time, in a single operation.
This is semantically equivalent to the `points3d_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_points3d_column_updates", spawn=True)
# Prepare a point cloud that evolves over 5 timesteps, changing the
# number of points in the process.
times = np.arange(10, 15, 1.0)
# fmt: off
positions = [
[1.0, 0.0, 1.0], [0.5, 0.5, 2.0],
[1.5, -0.5, 1.5], [1.0, 1.0, 2.5], [-0.5, 1.5, 1.0], [-1.5, 0.0, 2.0],
[2.0, 0.0, 2.0], [1.5, -1.5, 3.0], [0.0, -2.0, 2.5], [1.0, -1.0, 3.5],
[-2.0, 0.0, 2.0], [-1.5, 1.5, 3.0], [-1.0, 1.0, 3.5],
[1.0, -1.0, 1.0], [2.0, -2.0, 2.0], [3.0, -1.0, 3.0], [2.0, 0.0, 4.0],
]
# fmt: on
# At each timestep, all points in the cloud share the same but changing
# color and radius.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
radii = [0.05, 0.01, 0.2, 0.1, 0.3]
rr.send_columns(
"points",
indexes=[rr.TimeColumn("time", duration=times)],
columns=[
*rr.Points3D.columns(positions=positions).partition(
lengths=[2, 4, 4, 3, 4]
),
*rr.Points3D.columns(colors=colors, radii=radii),
],
)
```
Each row in the component column can be a batch of data, e.g. a batch of positions.
This lets you log the evolution of a point cloud over time efficiently.
### Updating a fixed number of arrows over time, in a single operation
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update a set of vectors over time.
See also the `arrows3d_column_updates` example, which achieves the same
thing in a single operation.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_arrows3d_row_updates", spawn=True)
# Prepare a fixed sequence of arrows over 5 timesteps.
# Origins stay constant, vectors change magnitude and direction, and each
# timestep has a unique color.
times = np.arange(10, 15, 1.0)
# At each time step, all arrows maintain their origin.
origins = np.linspace((-1, -1, 0), (1, 1, 0), 5)
vectors = [np.linspace((-1, -1, 0), (1, 1, i), 5) for i in range(5)]
# At each timestep, all arrows share the same but changing color.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
for i in range(5):
rr.set_time("time", duration=10 + i)
rr.log(
"arrows",
rr.Arrows3D(vectors=vectors[i], origins=origins, colors=colors[i]),
)
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update a set of vectors over time, in a single operation.
This is semantically equivalent to the `arrows3d_row_updates` example,
albeit much faster.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_arrows3d_column_updates", spawn=True)
# Prepare a fixed sequence of arrows over 5 timesteps.
# Origins stay constant, vectors change magnitude and direction, and each
# timestep has a unique color.
times = np.arange(10, 15, 1.0)
# At each time step, all arrows maintain their origin.
origins = [np.linspace((-1, -1, 0), (1, 1, 0), 5)] * 5
vectors = [np.linspace((-1, -1, 0), (1, 1, i), 5) for i in range(5)]
# At each timestep, all arrows share the same but changing color.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
rr.send_columns(
"arrows",
indexes=[rr.TimeColumn("time", duration=times)],
columns=[
*rr.Arrows3D.columns(origins=origins, vectors=vectors, colors=colors)
],
)
```
Each row in the component column can be a batch of data, e.g. a batch of positions.
This lets you log the evolution of a set of arrows over time efficiently.
### Updating a transform over time, in a single operation
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update a transform over time.
See also the `transform3d_column_updates` example, which achieves the same
thing in a single operation.
"""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_row_updates", spawn=True)
rr.set_time("tick", sequence=0)
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
rr.TransformAxes3D(10.0),
)
for t in range(100):
rr.set_time("tick", sequence=t + 1)
rr.log(
"box",
rr.Transform3D(
translation=[0, 0, t / 10.0],
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=truncated_radians(t * 4)
),
),
)
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update a transform over time, in a single operation.
This is semantically equivalent to the `transform3d_row_updates` example,
albeit much faster.
"""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_column_updates", spawn=True)
rr.set_time("tick", sequence=0)
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
rr.TransformAxes3D(10.0),
)
rr.send_columns(
"box",
indexes=[rr.TimeColumn("tick", sequence=range(1, 101))],
columns=rr.Transform3D.columns(
translation=[[0, 0, t / 10.0] for t in range(100)],
rotation_axis_angle=[
rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=truncated_radians(t * 4)
)
for t in range(100)
],
),
)
```
### Updating an image over time, in a single operation
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update an image over time.
See also the `image_column_updates` example, which achieves the same
thing in a single operation.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_image_row_updates", spawn=True)
for t in range(20):
rr.set_time("time", sequence=t)
image = np.zeros((200, 300, 3), dtype=np.uint8)
image[:, :, 2] = 255
image[50:150, (t * 10) : (t * 10 + 100)] = (0, 255, 255)
rr.log("image", rr.Image(image))
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update an image over time, in a single operation.
This is semantically equivalent to the `image_row_updates` example,
albeit much faster.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_image_column_updates", spawn=True)
# Timeline on which the images are distributed.
times = np.arange(0, 20)
# Create a batch of images with a moving rectangle.
width, height = 300, 200
images = np.zeros((len(times), height, width, 3), dtype=np.uint8)
images[:, :, :, 2] = 255
for t in times:
images[t, 50:150, (t * 10) : (t * 10 + 100), 1] = 255
# Log the ImageFormat and indicator once, as static.
format = rr.components.ImageFormat(
width=width, height=height, color_model="RGB", channel_datatype="U8"
)
rr.log("images", rr.Image.from_fields(format=format), static=True)
# Send all images at once.
rr.send_columns(
"images",
indexes=[rr.TimeColumn("step", sequence=times)],
# Reshape the images so `Image` can tell that this is several blobs.
#
# Note that the `Image` consumes arrays of bytes, so we should ensure
# that we take a uint8 view of it. This way, this also works when
# working with datatypes other than `U8`.
columns=rr.Image.columns(
buffer=images.view(np.uint8).reshape(len(times), -1)
),
)
```
### Updating custom user-defined values over time, in a single operation
[User-defined data](https://rerun.io/docs/howto/logging-and-ingestion/custom-data.md) can also benefit from the column-oriented APIs.
Consider this snippet, using the row-oriented `log` API:
```python
"""
Update custom user-defined values over time.
See also the `any_values_column_updates` example, which achieves the same
thing in a single operation.
"""
from __future__ import annotations
import math
import rerun as rr
rr.init("rerun_example_any_values_row_updates", spawn=True)
for step in range(64):
rr.set_time("step", sequence=step)
rr.log(
"/", rr.AnyValues(sin=math.sin(step / 10.0), cos=math.cos(step / 10.0))
)
```
which can be translated to the column-oriented `send_columns` API as such:
```python
"""
Update custom user-defined values over time, in a single operation.
This is semantically equivalent to the `any_values_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_any_values_column_updates", spawn=True)
timestamps = np.arange(0, 64)
rr.send_columns(
"/",
indexes=[rr.TimeColumn("step", sequence=timestamps)],
columns=rr.AnyValues.columns(
sin=np.sin(timestamps / 10.0), cos=np.cos(timestamps / 10.0)
),
)
```
# Clear out data using tombstones
In order to create coherent views of streaming data, the Rerun Viewer shows the latest values for each visible entity at the current timepoint. But some data may not be valid for the entire recording even if there are no updated values. How do you tell Rerun that something you've logged should no longer be shown?
## Log entities as cleared
The most straight forward option is to explicitly log that an entity has been cleared. Rerun allows you to do this by logging a special `Clear` to any path. The timepoint at which the `Clear` is logged is the time point after which that entity will no longer be visible in your views.
For example, if you have an object tracking application, your code might look something like this:
```python
β¦
for frame in sensors.read():
# Associate the following logs with `frame == frame.id`
rr.set_time("frame", sequence=frame.id)
# Do the actual tracking update
tracker.update(frame)
if tracker.is_lost:
# Clear everything on or below `tracked/{tracker.id}`
# and that happened on or before `frame == frame.id`
rr.log(f"tracked/{tracker.id}", rr.Clear(recursive=True))
else:
# Log data to the main entity and a child entity
rr.log(f"tracked/{tracker.id}", rr.Rect2D(tracker.bounds))
rr.log(f"tracked/{tracker.id}/cm", rr.Point2D(tracker.cm))
```
## Clarify data meaning
In some cases, the best approach may be to rethink how you log data to better express what is actually happening. Take the following example where update frequencies don't match:
```python
β¦
for frame in sensors.read():
# Associate the following logs with `frame = frame.id`
rr.set_time("frame", sequence=frame.id)
# Log every image that comes in
rr.log("input/image", rr.Image(frame.image))
if frame.id % 10 == 0:
# Run detection every 10 frames
detection = detector.detect(frame)
# Woops! These detections will not update at the
# same frequency as the input data and thus look strange
rr.log("input/detections", rr.Rect2D(detection.bounds))
```
You could fix this example by logging `rr.Clear`, but in this case it makes more sense to change what you log to better express what is happening. Re-logging the image to another namespace on only the frames where the detection runs makes it explicit which frame was used as the input to the detector. This will create a second view in the Viewer that always allows you to see the frame that was used for the current detection input.
Here is an example fix:
```python
class Detector:
β¦
def detect(self, frame):
downscaled = self.downscale(frame.image)
# Log the downscaled image
rr.log("detections/source", rr.Image(downscaled))
result = self.model(downscaled)
detection = self.post_process(result)
# Log the detections together with the downscaled image
# Image and detections will update at the same frequency
rr.log("downscaled/detections", rr.Rect2D(detection.bounds))
return detection
β¦
for frame in sensors.read():
# Associate the following logs with `frame = frame.id`
rr.set_time("frame", sequence=frame.id)
# Log every image that comes in
rr.log("input/image", rr.Image(frame.image))
if frame.id % 10 == 0:
# Run detection every 10 frames
# Logging of detections now happens inside the detector
detected = detector.detect(frame)
```
## Log data with spans instead of timepoints
In some cases you already know how long a piece of data will be valid at the time of logging. Rerun does **not yet support** associating logged data with spans like `(from_timepoint, to_timepoint)` or `(timepoint, time-to-live)`.
Follow the issue [here](https://github.com/rerun-io/rerun/issues/3008).
### Workaround by manually clearing entities
For now the best workaround is to manually clear data when it is no longer valid.
```python
# Associate the following data with `start_time` on the `time` timeline
rr.set_time("time", duration=start_time)
# Log the data as usual
rr.log("short_lived", rr.Tensor(one_second_tensor))
# Associate the following clear with `start_time + 1.0` on the `time` timeline
rr.set_time("time", duration=start_time + 1.0)
rr.log("short_lived", rr.Clear(recursive=False)) # or `rr.Clear.flat()`
# Set the time back so other data isn't accidentally logged in the future.
rr.set_time("time", duration=start_time)
```
# Using layers to append data to segments
In the [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md), datasets are a collection of segments, which are a collection of layers identified by a name.
Layers are immutable, but data can be added to segments by registering other layers with the same recording id but a different layer name.
This how-to page provides examples for two ways data can be added to existing datasets through layers.
> [!NOTE]
> Layers should not be confused with [MCAP decoders](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/decoders-explained.md), which serve a different purpose in the context of MCAP file ingestion.
## Adding data to existing segments using layers
When registering recordings to a dataset, the recordings are assigned the `"base"` layer name by default.
Let's register a few recordings from the [DROID](https://droid-dataset.github.io/) dataset (included in the Rerun repository for testing) to illustrate this:
```python
from pathlib import Path
import rerun as rr
sample_5_path = (
Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"
)
server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
client = server.client()
dataset = client.get_dataset(name="sample_dataset")
print(
dataset
.segment_table()
.select(
"rerun_segment_id",
"rerun_layer_names",
)
.sort("rerun_segment_id")
)
```
Output:
```
βββββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββ
β rerun_segment_id β rerun_layer_names β
β --- β --- β
β type: Utf8 β type: List[Utf8] β
βββββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββ‘
β ILIAD_50aee79f_2023_07_12_20h_55m_08s β [base] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_20_10h_40m_10s β [base] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_28_11h_25m_26s β [base] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββ€
β ILIAD_j807b3f8_2023_06_15_13h_42m_56s β [base] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββ€
β ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s β [base] β
βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββ
```
It is possible to append data to existing segments by creating new `.rrd` files with matching recording IDs, and registering them to the dataset under new layer names.
A common workflow is to query existing segment data, compute derived values (such as metrics or embeddings), and add them as new layers.
As an example, we use the dataset registered previously and compute the tracking error (L2 norm between commanded and actual joint positions) of the robotic arm:
```python
import numpy as np
from datafusion import col
# Query action (commanded) and observation (actual) joint positions
joints = dataset.filter_contents([
"/action/joint_positions",
"/observation/joint_positions",
]).reader(index="real_time")
# Compute tracking error: L2 norm of (commanded - actual) joint positions
segment_ids = pa.table(joints.select("rerun_segment_id").distinct())[
"rerun_segment_id"
].to_numpy()
rrd_paths = []
for seg_id in segment_ids:
# Filter to this segment and collect as a PyArrow table for efficient
# extraction to NumPy
segment_data = pa.table(
joints.filter(col("rerun_segment_id") == seg_id).select(
"real_time",
"/action/joint_positions:Scalars:scalars",
"/observation/joint_positions:Scalars:scalars",
)
)
timestamps = segment_data["real_time"].to_numpy()
actions = np.vstack(
segment_data["/action/joint_positions:Scalars:scalars"].to_numpy()
)
observations = np.vstack(
segment_data["/observation/joint_positions:Scalars:scalars"].to_numpy()
)
# Compute L2 tracking error per timestep
tracking_error = np.linalg.norm(actions - observations, axis=1)
# Create derived RRD with tracking error timeline
rrd_path = TMP_DIR / f"{seg_id}_tracking_error.rrd"
rrd_paths.append(rrd_path)
with rr.RecordingStream(
application_id="rerun_example_tracking_error", recording_id=seg_id
) as rec:
rec.save(rrd_path)
rr.send_columns(
"/derived/tracking_error",
indexes=[rr.TimeColumn("real_time", timestamp=timestamps)],
columns=rr.Scalars.columns(scalars=tracking_error),
)
# Register derived RRDs as a new layer
dataset.register(
[p.as_uri() for p in rrd_paths], layer_name="tracking_error"
).wait()
```
The key steps are:
1. Query action (commanded) and observation (actual) joint positions from the dataset
2. For each segment, compute the L2 norm of the difference as tracking error
3. Create a new `.rrd` file with the same `recording_id` as the original segment
4. Log the derived data using `send_columns()` for efficient columnar logging
5. Register all derived `.rrd` files to the dataset with a `"tracking_error"` layer name
The `"rerun_layer_names"` column of the segment table confirms the new layer was added:
```python
segment_table = (
dataset
.segment_table()
.select(
"rerun_segment_id",
"rerun_layer_names",
)
.sort("rerun_segment_id")
)
print(segment_table)
```
Output:
```
βββββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββ
β rerun_segment_id β rerun_layer_names β
β --- β --- β
β type: Utf8 β type: List[Utf8] β
βββββββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββ‘
β ILIAD_50aee79f_2023_07_12_20h_55m_08s β [base, tracking_error] β
βββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_20_10h_40m_10s β [base, tracking_error] β
βββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_28_11h_25m_26s β [base, tracking_error] β
βββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββ€
β ILIAD_j807b3f8_2023_06_15_13h_42m_56s β [base, tracking_error] β
βββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββ€
β ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s β [base, tracking_error] β
βββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββ
```
For another example of computing derived data, see the [Query video streams](https://rerun.io/docs/howto/query-and-transform/query_videos.md) page.
In the next section, we will demonstrate that the data has indeed been added by querying the dataset again.
## Adding properties to segments using layers
In addition to regular Rerun data, layers can be used to add [properties](https://rerun.io/docs/concepts/query-and-transform/properties-and-segments.md) to segments.
This is useful for tagging segments with derived metadata based on their content.
In this example, we query the tracking error computed in the previous section, calculate the mean error per segment, and create a `tracking_good` boolean property based on a threshold:
```python
# Query the tracking error we just added and compute a quality metric
from datafusion import functions as F
tracking = dataset.filter_contents(["/derived/tracking_error"]).reader(
index="real_time"
)
quality_stats = pa.table(
tracking
.aggregate(
col("rerun_segment_id"),
[
F.avg(col("/derived/tracking_error:Scalars:scalars")[0]).alias(
"mean_error"
)
],
)
.with_column("tracking_good", col("mean_error") < 0.13)
.select("rerun_segment_id", "tracking_good")
)
# Create RRDs with just the property
rrd_paths = []
for seg_id, tracking_good in zip(
quality_stats["rerun_segment_id"], quality_stats["tracking_good"]
):
rrd_path = TMP_DIR / f"{seg_id}_quality.rrd"
rrd_paths.append(rrd_path)
with rr.RecordingStream(
application_id="rerun_example_quality", recording_id=seg_id
) as rec:
rec.save(rrd_path)
rec.send_property("quality", rr.AnyValues(tracking_good=tracking_good))
# Register as a separate layer
dataset.register([p.as_uri() for p in rrd_paths], layer_name="quality").wait()
```
The key steps are:
1. Query the derived tracking error data we just added
2. Use DataFusion's `aggregate()` to compute the mean error per segment
3. Threshold the mean to create a boolean `tracking_good` property
4. Create new `.rrd` files with `send_property()` to log the property
5. Register under a separate `"quality"` layer
The property now appears in the segment table:
```python
# The segment table now shows both layers and the derived property
segment_table = (
dataset
.segment_table()
.select(
"rerun_segment_id",
"rerun_layer_names",
"property:quality:tracking_good",
)
.sort("rerun_segment_id")
)
print(segment_table)
```
Output:
```
βββββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ
β rerun_segment_id β rerun_layer_names β property:quality:tracking_good β
β --- β --- β --- β
β type: Utf8 β type: List[Utf8] β type: nullable List[nullable bool] β
β β β component: tracking_good β
β β β entity_path: /__properties/quality β
β β β kind: data β
βββββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββ‘
β ILIAD_50aee79f_2023_07_12_20h_55m_08s β [base, tracking_error, quality] β [false] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_20_10h_40m_10s β [base, tracking_error, quality] β [false] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_28_11h_25m_26s β [base, tracking_error, quality] β [true] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ€
β ILIAD_j807b3f8_2023_06_15_13h_42m_56s β [base, tracking_error, quality] β [true] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ€
β ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s β [base, tracking_error, quality] β [true] β
βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ
```
See [this page](https://rerun.io/docs/concepts/query-and-transform/properties-and-segments.md) for a deep dive into properties and segment tables.
## FAQ
### Can I modify a layer after it has been registered?
No.
Layers are immutable.
### What happens if I register to an existing layer name?
Registering a `.rrd` file with a `recording_id` and `layer_name` that already exists will result in an error.
### How can I replace an existing layer?
There is currently no way to replace an existing layer using the Python SDK.
The current workaround consists of recreating the dataset.
### Can I query a single layer with the dataframe query?
No.
Segments are considered an aggregation of all their layers.
There is currently no way to query data from a single layer only.
### Must layers be registered to all segments in a dataset?
No.
Each segment can have its own set of layers.
Some segments may have additional layers that others do not.
### What is the default layer name?
When you register a recording without specifying a `layer_name`, it is assigned to the `"base"` layer.
### Is it possible to obtain a dataframe with a list of all layers in a dataset?
Yes.
The [`DatasetEntry.segment_table()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry.segment_table) method returns a DataFusion DataFrame with one row per segment and a `rerun_layer_names` column listing the layers of each segment:
```python
layers = (
dataset
.segment_table()
.select(
"rerun_segment_id",
"rerun_layer_names",
)
.sort("rerun_segment_id")
)
print(layers)
```
Output:
```
βββββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ
β rerun_segment_id β rerun_layer_names β
β --- β --- β
β type: Utf8 β type: List[Utf8] β
βββββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββ‘
β ILIAD_50aee79f_2023_07_12_20h_55m_08s β [base, tracking_error, quality] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_20_10h_40m_10s β [base, tracking_error, quality] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
β ILIAD_5e938e3b_2023_07_28_11h_25m_26s β [base, tracking_error, quality] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
β ILIAD_j807b3f8_2023_06_15_13h_42m_56s β [base, tracking_error, quality] β
βββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββ€
β ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s β [base, tracking_error, quality] β
βββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ
```
# Send user-defined data
Rerun comes with many pre-built [Types](https://rerun.io/docs/reference/types.md) that you can use out of the box. As long as your own data can be decomposed into Rerun [components](https://rerun.io/docs/reference/types/components.md) or can be serialized with [Apache Arrow](https://arrow.apache.org/), you can log it directly without needing to recompile Rerun.
For Python and Rust we have helpers for this, called `AnyValues`, allowing you to easily attach custom values to any entity instance.
For C++ a similar thing can be accomplished without the helpers.
You find the documentation for these helpers here:
- [`AnyValues` in Python](https://ref.rerun.io/docs/python/main/common/custom_data/#rerun.AnyValues)
- [`AnyValues` in Rust](https://docs.rs/rerun/latest/rerun/struct.AnyValues.html)
```python
"""Log arbitrary data."""
import rerun as rr
rr.init("rerun_example_any_values", spawn=True)
rr.log(
"any_values",
rr
.AnyValues(
# Using arbitrary Arrow data.
homepage="https://www.rerun.io",
repository="https://github.com/rerun-io/rerun",
)
# Using Rerun's builtin components.
.with_component_override(
"confidence", rr.components.ScalarBatch._COMPONENT_TYPE, [1.2, 3.4, 5.6]
)
.with_component_override(
"description", rr.components.TextBatch._COMPONENT_TYPE, "Bla bla blaβ¦"
),
)
```
If your values should be grouped together and that grouping isn't referred to from many places that need to stay aligned we have a helpers for this called, `DynamicArchetype` which adds some structural grouping to multiple values.
You find the documentation for these helpers here:
- [`DynamicArchetype` in Python](https://ref.rerun.io/docs/python/main/common/custom_data/#rerun.DyanamicArchetype)
- [`DynamicArchetype` in Rust](https://docs.rs/rerun/latest/rerun/struct.DynamicArchetype.html)
```python
"""Log arbitrary archetype data."""
import rerun as rr
rr.init("rerun_example_dynamic_archetype", spawn=True)
rr.log(
"new_archetype",
rr
.DynamicArchetype(
archetype="MyArchetype",
components={
# Using arbitrary Arrow data.
"homepage": "https://www.rerun.io",
"repository": "https://github.com/rerun-io/rerun",
},
)
# Using Rerun's builtin components.
.with_component_override(
"confidence", rr.components.ScalarBatch._COMPONENT_TYPE, [1.2, 3.4, 5.6]
)
.with_component_override(
"description", rr.components.TextBatch._COMPONENT_TYPE, "Bla bla blaβ¦"
),
)
```
You can also create your own component by implementing the `AsComponents` [Python protocol](https://ref.rerun.io/docs/python/0.9.0/common/interfaces/#rerun.AsComponents) or [Rust trait](https://docs.rs/rerun/latest/rerun/trait.AsComponents.html), which means implementing the function, `as_component_batches()`.
## Remapping to a Rerun archetype
Let's start with a simple example where you have your own point cloud class that is perfectly representable as a Rerun archetype.
```python
@dataclass
class LabeledPoints:
points: np.ndarray
labels: List[str])
```
If you implement `as_component_batches()` on `LabeledPoints`, you can pass it directly to `rr.log`. The simplest possible way is to use the matching Rerun archetypeβs `as_component_batches` method.
```python
import rerun as rr # pip install rerun-sdk
@dataclass
class LabeledPoints:
points: np.ndarray
labels: List[str]
def as_component_batches(self) -> list[rr.ComponentBatch]:
return rr.Points3D(positions=self.points,
labels=self.labels).as_component_batches()
β¦
# Somewhere deep in your code
classified = my_points_classifier(β¦) # type: LabeledPoints
rr.log("points/classified", classified)
```
## Custom archetypes and components
You can also define and log your own custom archetypes and components completely from user code, without rebuilding Rerun.
In this example we extend the Rerun Points3D archetype with a custom confidence component and user-defined archetype.
This is what it looks like in the in the dataframe view:
```python
"""Shows how to implement custom archetypes and components."""
from __future__ import annotations
import argparse
from typing import Any
import numpy as np
import numpy.typing as npt
import pyarrow as pa
import rerun as rr
class ConfidenceBatch(rr.ComponentBatchMixin): # type: ignore[misc]
"""A batch of confidence data."""
def __init__(self: Any, confidence: npt.ArrayLike) -> None:
self.confidence = confidence
def as_arrow_array(self) -> pa.Array:
"""The arrow batch representing the custom component."""
return pa.array(self.confidence, type=pa.float32())
class CustomPoints3D(rr.AsComponents): # type: ignore[misc]
"""A custom archetype extending the builtin `Points3D` with extra data."""
def __init__(
self: Any, positions: npt.ArrayLike, confidences: npt.ArrayLike
) -> None:
self.points3d = rr.Points3D(positions)
self.confidences = ConfidenceBatch(confidences).described(
rr.ComponentDescriptor(
"user.CustomPoints3D:confidences",
archetype="user.CustomPoints3D",
component_type="user.Confidence",
)
)
def as_component_batches(self) -> list[rr.DescribedComponentBatch]:
return [
# The components from Points3D
*self.points3d.as_component_batches(),
# Custom confidence data
self.confidences,
]
def log_custom_data() -> None:
lin = np.linspace(-5, 5, 3)
z, y, x = np.meshgrid(lin, lin, lin, indexing="ij")
point_grid = np.vstack([x.flatten(), y.flatten(), z.flatten()]).T
rr.log(
"left/my_confident_point_cloud",
CustomPoints3D(
positions=point_grid,
confidences=[42],
),
)
rr.log(
"right/my_polarized_point_cloud",
CustomPoints3D(
positions=point_grid, confidences=np.arange(0, len(point_grid))
),
)
def main() -> None:
parser = argparse.ArgumentParser(
description="Logs rich data using the Rerun SDK."
)
rr.script_add_args(parser)
args = parser.parse_args()
rr.script_setup(args, "rerun_example_custom_data")
log_custom_data()
rr.script_teardown(args)
if __name__ == "__main__":
main()
```
## Creating/augmenting visualizations from custom data
All components can be mapped to arbitrary slots of visualizers.
For a general information on component mapping see [component mappings](https://rerun.io/docs/howto/visualization/component-mappings.md),
for the common case of plotting see [plot any scalar](https://rerun.io/docs/howto/visualization/plot-any-scalar.md)
> [!INFO]
> Complex mappings e.g. from scalars to colors are not yet possible, but will be supported in future versions
> by exposing more functionality from [Lenses](https://rerun.io/docs/concepts/query-and-transform/lenses.md) directly in the Viewer.
# Share recordings across multiple processes
A common need is to log data from multiple processes and then visualize all of that data as part of a single shared recording.
Rerun has the notion of a [Recording ID](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) for that: any recorded datasets that share the same Recording ID will be visualized as one shared dataset.
The data can be logged from any number of processes, whether they run on the same machine or not, or implemented in different programming languages.
All that matter is that they share the same Recording ID.
By default, Rerun generates a random Recording ID everytime you start a new logging session, but you can override that behavior, e.g.:
```python
rr.init("rerun_example_shared_recording", recording_id="my_shared_recording")
```
It's up to you to decide where each recording ends up:
- all processes could stream their share of the data in real-time to a Rerun Viewer,
- or maybe they all write to their own file on disk that are later loaded in a viewer,
- or some other combination of the above.
Here's a simple example of such a workflow:
```python
# Process 1 logs some spheres to a recording file.
./app1.py # rr.init(recording_id='my_shared_recording', rr.save('/tmp/recording1.rrd')
# Process 2 logs some cubes to another recording file.
./app2.py # rr.init(recording_id='my_shared_recording', rr.save('/tmp/recording2.rrd')
# Visualize a 3D scene with both spheres and cubes.
rerun /tmp/recording*.rrd # they share the same Recording ID!
```
For more information, check out our dedicated examples:
* [π Python](https://github.com/rerun-io/rerun/blob/latest/examples/python/shared_recording/shared_recording.py)
* [π¦ Rust](https://github.com/rerun-io/rerun/blob/latest/examples/rust/shared_recording/src/main.rs)
* [π C++](https://github.com/rerun-io/rerun/blob/latest/examples/cpp/shared_recording/main.cpp)
### Merging recordings with the Rerun CLI
It is possible to merge multiple recording files into a single one using the [Rerun CLI](https://rerun.io/docs/reference/cli.md), e.g. `rerun rrd merge -o merged_recordings.rrd my_first_recording.rrd my_second_recording.rrd`.
The Rerun CLI offers several options to manipulate recordings in different ways, check out [the CLI reference](https://rerun.io/docs/reference/cli.md) for more information.
# Working with MCAP
The Rerun Viewer has built-in support for opening [MCAP](https://mcap.dev/) files, an open container format for storing timestamped messages.
## Supported message formats
Here's a quick summary of Rerun's MCAP importer:
* Automatic conversion to Rerun archetypes is supported for common ROS 2 & Foxglove messages.
* Other ROS 2 & Foxglove messages are decoded into queryable components via [reflection](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md).
For a detailed overview of Rerun's built-in MCAP support, please refer to our [Supported Message Formats](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md) page.
We are continually expanding the supported MCAP message types and are [interested in your feedback](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md).
## Quick start
### Loading MCAP files
The simplest way to get started is to load an MCAP file directly:
```bash
# View an MCAP file in the Rerun Viewer
rerun your_data.mcap
```
You can also drag and drop MCAP files into the Rerun Viewer or load them using the SDK:
```python
"""Load an MCAP file using the Python SDK."""
import sys
import rerun as rr
path_to_mcap = sys.argv[1]
# Initialize the SDK and give our recording a unique name
rr.init("rerun_example_load_mcap", spawn=True)
# Load the MCAP file
rr.log_file_from_path(path_to_mcap)
recording = rr.get_data_recording()
assert recording is not None
recording.flush()
```
### Basic conversion
Convert MCAP files to Rerun's native format for faster loading:
```bash
# Convert MCAP to RRD format for faster loading
rerun mcap convert input.mcap -o output.rrd
# View the converted file
rerun output.rrd
```
## Data model
Rerun's data model is based on an [entity component system (ECS)](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) that is a bit different to the message-based model of [MCAP](https://mcap.dev).
To map MCAP messages to Rerun entities we make the following assumptions:
* MCAP topics corresponds to Rerun entities.
* Messages from the same topic within an MCAP chunk will be placed into a corresponding [Rerun chunk](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md).
* The contents of an MCAP message will be extracted to Rerun components and grouped under a corresponding Rerun archetype.
* `message_log_time` and `message_publish_time` of an MCAP message will be carried over to Rerun as two distinct [timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md).
### Layered architecture
Rerun uses a _layered architecture_ to process MCAP files at different levels of abstraction. This design allows the same MCAP file to be ingested in multiple ways simultaneously, from raw bytes to semantically meaningful visualizations.
Each layer extracts different types of information from the MCAP source and each of the following layers will create distinct Rerun archetypes:
- **`raw`**: Logs the unprocessed message bytes as Rerun blobs without any interpretation
- **`schema`**: Extracts metadata about channels, topics, and schemas
- **`stats`**: Extracts file-level metrics like message counts, time ranges, and channel statistics into `__mcap_properties` in the RRD
- **`metadata`** Extracts metadata records (if present) into `__mcap_metadata` in the RRD
- **`attachments`**: Extracts MCAP attachment records (if present) as static data under `__mcap_attachments`
- **`protobuf`**: Automatically decodes protobuf-encoded messages using reflection
- **`ros2msg`**: Provides semantic conversion of common ROS2 message types into Rerun's visualization components
- **`ros2_reflection`**: Automatically decodes ROS2 messages using reflection
- **`recording_info`**: Extracts recording metadata such as message counts, start time, and session information into `__mcap_properties` in the RRD
- **`urdf`**: Uses Rerun's built-in URDF loader when a ROS 2 `/robot_description` string topic is present
By default, Rerun analyzes an MCAP file to determine which decoders are active to provide the most comprehensive view of your data, while avoiding duplication.
You can also choose to activate only specific decoders that are relevant to your use case.
The following shows how to select specific decoders:
```sh
# Use only specific decoders
rerun mcap convert input.mcap -d protobuf -d stats -o output.rrd
# Use multiple decoders for different perspectives
rerun mcap convert input.mcap -d ros2msg -d raw -d recording_info -o output.rrd
# Add robot geometry from robot_description topics
rerun mcap convert input.mcap -d ros2msg -d urdf -o output.rrd
```
For a detailed explanation of how each decoder works and when to use them, see [Decoders Explained](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/decoders-explained.md).
## Advanced usage
For advanced command-line options and automation workflows, see the [CLI Reference](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/cli-reference.md) for complete documentation of all available commands and flags.
# Send tables to Rerun
> [!NOTE]
> The `send_table` API is currently experimental and may change in future releases.
Rerun now supports sending tabular data to the Rerun Viewer! This feature allows you to visualize and interact with dataframes (encoded as Arrow record batches) directly in the Rerun Viewer environment.
## Overview
The `send_table` API provides a straightforward way to send tabular data to the Rerun Viewer. This is particularly useful for:
- Inspecting dataframes alongside other visualizations
- Debugging data processing pipelines
- Presenting structured data in a readable format
## References
For complete examples of using `send_table`, please refer to:
- [π Jupyter Notebook](https://github.com/rerun-io/rerun/blob/main/examples/notebook/notebook/send_table.ipynb)
- [π Python SDK](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/howto/send_table.py)
## Prerequisites
- Rerun SDK (Python)
- PyArrow library
- Pandas
- NumPy
Which can be installed via:
```sh
pip install rerun-sdk[notebook] pyarrow pandas numpy
```
## Basic usage
### Connecting to the Viewer
```python
from rerun.experimental import ViewerClient
# Connect to a running Rerun Viewer
client = ViewerClient.connect(url="rerun+http://127.0.0.1:9876/proxy")
```
### Sending a simple table
```python
import pyarrow as pa
# Create a record batch from a Python dictionary
record_batch = pa.RecordBatch.from_pydict({
"id": [1, 2, 3],
"url": ["https://www.rerun.io", "https://github.com/rerun-io/rerun", "https://crates.io/crates/rerun"],
})
# Send the table to the viewer with an identifier
client.send_table("My Table", record_batch)
```
### Using with Pandas dataframes
You can also send Pandas dataframes by converting them to a record batch:
```python
import pandas as pd
import numpy as np
import pyarrow as pa
# Create a sample DataFrame
dates = pd.date_range("20230101", periods=6)
df = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list("ABCD"))
# Convert to record batch and send to the viewer.
client.send_table("Pandas DataFrame", pa.RecordBatch.from_pandas(df))
```
## Using in Jupyter notebooks
Rerun provides special support for Jupyter notebooks, you can find more information here: [https://rerun.io/docs/howto/integrations/embed-notebooks]
Note that this API makes use of `rr.notebook.Viewer`:
```python
import rerun as rr
import pyarrow as pa
# For inline display
os.environ["RERUN_NOTEBOOK_ASSET"] = "inline"
# Create and display the viewer
viewer = rr.notebook.Viewer(width="auto", height="auto")
viewer.display()
# Send table directly to the inline viewer
viewer.send_table(
"My Table",
pa.RecordBatch.from_pydict({"Column A": [1, 2, 3], "Column B": ["https://www.rerun.io", "Hello", "World"]}),
)
```
You can also use the native viewer instead of the inline viewer:
```python
os.environ["RERUN_NOTEBOOK_ASSET"] = "serve-local"
# Connect to a running Rerun Viewer
client = ViewerClient.connect(url="rerun+http://127.0.0.1:9876/proxy")
```
## Current limitations
As this is an experimental API, there are several limitations to be aware of:
- Only a single record batch is supported per table
- Tables can't be saved/loaded from files yet (unlike `.rrd` files for recordings)
- Integration with the rest of the Rerun API is still in progress
- Rust and C++ support will be added after the API stabilizes
- The API may undergo significant changes as we iterate based on user feedback
## What's next
The `send_table` API is still evolving and we plan to tackle all of the limitations mentioned above.
We welcome your [feedback and suggestions](https://rerun.io/feedback) as we continue to improve this feature!
# Optimize chunk count
## Understanding chunks and their impact on performance
Rerun stores all its data in Chunks β Arrow-encoded tables of data. A basic understanding of chunks is key to understanding how Rerun works and why performance behaves the way it does. See the [chunk concept documentation](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) for more details.
Chunks are the atomic unit of work in Rerun: the performance cost of logging, ingesting, storing, querying, and visualizing data (including memory overhead to some extent) scales roughly linearly with the number of chunks in the system (ignoring caching and indexing optimizations). Many small chunks cause significantly more overhead than fewer, larger ones. Larger chunks reduce index pressure and per-chunk overhead, improving write and query throughput.
Rerun provides several online and offline systems to track and reduce chunk count. Using them effectively can yield dramatic performance improvements in real-world scenarios.
The process of merging smaller chunks into fewer, larger ones is called compaction. It occurs at multiple points across the dataβs lifecycle, each with different constraints. Earlier compaction reduces the work needed later in the pipeline.
Each of these stages is defined by constraints such as:
* Where it runs (e.g. client-side, server-side, standalone?)
* What data it can access (e.g. partial data or full recording? streaming data or random access possible?)
* What compute resources it can use (e.g. soft real-time or offline?)
## SDK micro-batching
Micro-batching is an online compaction mechanism on the SDK side that compacts small log calls into larger chunks before sending. A background thread flushes these batches either at fixed intervals or when they reach a size threshold.
This reduces metadata overhead (fewer chunks), which improves network and CPU efficiency. By default, the SDK flushes:
* every ~200β―ms when logging to file,
* every ~8β―ms when logging to the Rerun Viewer directly, or
* when the batch reaches ~1β―MiB.
These defaults aim to balance latency and throughput. To adjust them, see the [micro-batching documentation](https://rerun.io/docs/reference/sdk/micro-batching.md).
Micro-batching trades a bit of latency for significantly fewer chunks, improving ingestion throughput and downstream performance. While lightweight to compute, it operates with minimal context β all it sees is a small rolling window of logs β so compaction is far from optimal.
Constraints:
* Runs: client-side, in the SDK
* Data access: only a short rolling window of recent logs
* Operational limits: minimal CPU and memory usage to avoid impacting the host process
## In-viewer compaction
On the server side, the Rerun Viewer performs continuous, online compaction in the Chunk Store. As data arrives, smaller chunks are merged until they reach target sizes, preventing an explosion of tiny chunks. Triggers are based on row count and byte thresholds, similar to SDK micro-batching.
By default, chunks compact up to ~384β―KiB, or ~4096 rows (or 1024 for unsorted time chunks).
These settings balance ingestion speed, query performance, and memory use. You can configure them using environment variables such as `RERUN_CHUNK_MAX_BYTES` and `RERUN_CHUNK_MAX_ROWS`. See the [store compaction docs](https://rerun.io/docs/reference/store-compaction.md) for more.
Viewer-side compaction is more expensive than SDK-side micro-batching but has access to full context, enabling much more effective decisions. Fortunately, the cost is kept low thanks to micro-batching upstream: the better the batching in the SDK, the less work needed in the Viewer (as we'll see below, the CLI can even make that work disappear entirely!).
Constraints:
* Runs: server-side, in the Viewer
* Data access: the full in-memory dataset (although older data may have been [garbage collected](https://rerun.io/docs/howto/visualization/limit-ram.md))
* Operational limits: must remain lightweight and responsive, as it shares CPU with other real-time viewer workloads. Runs as a streaming process β compaction happens as data arrives.
## Inspecting and compacting chunks with the Rerun CLI
Rerun offers CLI tools to inspect and optimize .rrd recordings or streamed data files.
Use [`rerun rrd stats`](https://rerun.io/docs/reference/cli.md) to view stats like chunk counts, sizes, and row distributions. This helps you determine if compaction is needed. For example:
```sh
$ rerun rrd stats <(curl 'https://app.rerun.io/version/latest/examples/nuscenes_dataset.rrd')
Overview
----------
num_chunks = 576
num_entity_paths = 52
num_chunks_without_components = 0 (0.000%)
num_rows = 1β563
num_rows_min = 1
num_rows_max = 101
num_rows_avg = 2.714
num_static = 46
num_indexes_min = 0
num_indexes_max = 3
num_indexes_avg = 2.760
num_components_min = 1
num_components_max = 10
num_components_avg = 1.988
Size (schema + data, uncompressed)
----------------------------------
ipc_size_bytes_total = 112 MiB
ipc_size_bytes_min = 1.4 KiB
ipc_size_bytes_max = 568 KiB
ipc_size_bytes_avg = 200 KiB
ipc_size_bytes_p50 = 161 KiB
ipc_size_bytes_p90 = 567 KiB
ipc_size_bytes_p95 = 567 KiB
ipc_size_bytes_p99 = 568 KiB
ipc_size_bytes_p999 = 568 KiB
# β¦ truncated β¦
```
If a file contains many small chunks, run [`rerun rrd optimize`](https://rerun.io/docs/reference/cli.md) to rewrite it with fewer, larger chunks. For example:
```sh
$ rerun rrd optimize --max-size 2MiB -o nuscenes_compacted.rrd <(curl 'https://app.rerun.io/version/latest/examples/nuscenes_dataset.rrd')
merge/compaction finished srcs=["/dev/fd/63"] time=2.51217062s num_chunks_before=576 num_chunks_after=217 num_chunks_reduction="-62.326%" srcs_size_bytes=90.0 MiB dst_size_bytes=89.6 MiB size_reduction="-0.474%"
$ rrd stats nuscenes_compacted.rrd
Overview
----------
num_chunks = 278
num_entity_paths = 52
num_chunks_without_components = 0 (0.000%)
num_rows = 1β084
num_rows_min = 1
num_rows_max = 101
num_rows_avg = 3.899
num_static = 23
num_indexes_min = 0
num_indexes_max = 3
num_indexes_avg = 2.752
num_components_min = 1
num_components_max = 10
num_components_avg = 2.133
Size (schema + data, uncompressed)
----------------------------------
ipc_size_bytes_total = 111 MiB
ipc_size_bytes_min = 1.7 KiB
ipc_size_bytes_max = 1.0 MiB
ipc_size_bytes_avg = 410 KiB
ipc_size_bytes_p50 = 567 KiB
ipc_size_bytes_p90 = 670 KiB
ipc_size_bytes_p95 = 713 KiB
ipc_size_bytes_p99 = 838 KiB
ipc_size_bytes_p999 = 1.0 MiB
# β¦ truncated β¦
```
This produces a new file where chunks have been merged up to the size and row thresholds of the selected optimization profile (see below) (further capped by `--max-size 2MiB` in the example above). This significantly reduces viewer-side load and improves performance for future queries and visualization.
Because it runs offline, the CLI compactor has full access to the dataset and no real-time constraints, making it the most effective tool for optimal compaction. It's a good idea to compact files ahead of time if theyβll be queried or visualized repeatedly.
> [!WARNING]
> `rerun rrd optimize` will automatically migrate the data to the latest version of the RRD protocol, if needed.
Note that `rerun rrd optimize` ships two preset profiles, selected with `--profile`, that set sensible thresholds for two common targets:
* `object-store` *(default)* β large chunks (up to ~65k rows, ~2β―MiB), tuned for object-store-backed datasets stored on catalog servers, where query throughput and network streaming matter most.
* `live` β small chunks (up to ~4096 rows, ~384β―KiB), tuned for the live-Viewer workflow where the time panel benefits from finer-grained resolution.
Per-knob flags (`--max-rows`, `--max-size`, β¦) and the `RERUN_CHUNK_MAX_*` environment variables override the profile's values.
Constraints:
* Runs: standalone CLI tool
* Data access: full dataset (must fit in memory)
* Operational limits: none -- runs fully offline
## Compacting chunks with the chunk processing API
The same compaction logic that powers `rerun rrd optimize` is exposed in the [Chunk Processing API](https://rerun.io/docs/concepts/logging-and-ingestion/chunk-processing-api.md), so you can fold optimization into a Python ingestion or conversion pipeline rather than running it as a separate CLI step:
```python
(
McapReader(mcap_path)
.stream()
.collect(optimize=OptimizationProfile.OBJECT_STORE)
.write_rrd(
output_path,
application_id="rerun_example_optimize",
recording_id=mcap_path.stem,
)
)
```
[`LazyChunkStream.collect()`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyChunkStream) materializes the pipeline into a `ChunkStore`; passing an `OptimizationProfile` runs extra compaction passes tuned for a specific target. The two presets mirror the CLI's `--profile` values:
* `OptimizationProfile.OBJECT_STORE` (corresponds to `--profile object-store`, the CLI default) β large chunks for object-store-backed datasets;
* `OptimizationProfile.LIVE` (corresponds to `--profile live`) β small chunks for the live-Viewer workflow.
* **Note:** `collect()` materializes the entire pipeline into an in-memory `ChunkStore` before writing, so the full recording must fit in RAM.
## Conclusion
* Compaction isnβt a minor optimization β it can and frequently yields massive performance gains depending on your workload.
* Rerun applies micro-batching and compaction by default, but optimal settings vary per use case.
* Compaction can (and should) happen at multiple stages, each with different tradeoffs, operating under very different constraints.
* Once data has been recorded, two complementary tools let you preemptively optimize it for downstream use:
* The Rerun CLI: `rerun rrd stats` to diagnose, `rerun rrd optimize` for one-shot offline compaction.
* The [Chunk Processing API](https://rerun.io/docs/concepts/logging-and-ingestion/chunk-processing-api.md): same compaction logic, exposed in-process so you can fold it into a Python ingestion or conversion pipeline via `collect(optimize=OptimizationProfile.β¦)`.
# Loading URDF models
Rerun features a built-in [importer](https://rerun.io/docs/concepts/logging-and-ingestion/importers/overview) for [URDF](https://en.wikipedia.org/wiki/URDF) files.
## Overview
Using a `URDF` in Rerun only requires you to load the file with the logging API.
This will automatically invoke the importer, which will take care of:
* resolving paths to meshes
* loading meshes and shapes as Rerun entities
* loading the joint transforms and associated frame IDs of links
Once that is done, the joints can be updated by sending [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)s, where you have to set the `parent_frame` and `child_frame` fields explicitly to each joint's specific frame IDs.
> [!NOTE]
> Previous versions (< 0.28) required you to send transforms with _implicit_ frame IDs, i.e. having to send each joint transform on a specific entity path.
> This was dropped in favor of _named_ frame IDs, which is more in line with ROS and allows you to send all transform updates on one entity (e.g. a `transforms` entity like in the example below).
## Example
Here is an example that demonstrates how to load and update a `URDF` with the Python SDK:
```python
from pathlib import Path
import rerun as rr
from rerun import RecordingStream
with RecordingStream("rerun_example_load_urdf") as rec:
rec.spawn()
# `log_file_from_path` automatically uses the built-in URDF importer.
urdf_path = Path(__file__).parent / "minimal.urdf"
rec.log_file_from_path(urdf_path, static=True)
# The `flush` call is optional, but it helps with logging consistency,
# because it ensures that the URDF finishes loading before continuing.
rec.flush()
# Later, in your logging code, you'll update the joints using transforms.
# A minimal example for updating a revolute joint that connects two links:
joint_axis = [0, 0, 1] # comes from URDF
joint_angle = 1.216 # radians
origin_xyz = [0, 0, 0.1] # comes from URDF
# Make sure that `parent_frame` and `child_frame` match the joint's
# frame IDs in the URDF file.
rec.log(
"transforms",
rr.Transform3D(
rotation=rr.RotationAxisAngle(axis=joint_axis, angle=joint_angle),
translation=origin_xyz,
parent_frame="base_link",
child_frame="child_link",
),
)
```
For a full animation example, see the [Python animated URDF example](https://github.com/rerun-io/rerun/tree/main/examples/python/animated_urdf). There's also a [Rust example](https://github.com/rerun-io/rerun/tree/main/examples/rust/animated_urdf).
## URDF utilities (Python)
Rerun provides the [`rr.urdf`](https://github.com/rerun-io/rerun/tree/main/rerun_py/rerun_sdk/rerun/urdf.py) Python module that can facilitate the handling of URDF models in your code.
It can be used as an alternative to other 3rd-party packages like [yourdfpy](https://yourdfpy.readthedocs.io/en/latest/index.html) or [pytransforms3d](https://dfki-ric.github.io/pytransform3d/index.html).
As shown below, you can use it e.g. to access individual joints of the URDF model and to compute their respective transforms based on joint states (e.g. angles for revolute joints).
These transforms can be directly sent to Rerun.
### UrdfTree
Load a URDF file and access its structure:
```python
urdf_tree = rr.urdf.UrdfTree.from_file_path("robot.urdf", entity_path_prefix=None)
# Access properties
robot_name = urdf_tree.name
root_link = urdf_tree.root_link()
joints = urdf_tree.joints()
# Lookup by name
urdf_tree.get_joint_by_name("shoulder")
urdf_tree.get_link_by_name("base_link")
# Get the entity paths of collision or visual geometries of a link.
# This can be used for example to update the color / transparency during runtime:
for visual_path in urdf_tree.get_visual_geometry_paths("gripper"):
rec.log(visual_path, rr.Asset3D.from_fields(albedo_factor=[255, 0, 0, 100]), static=True)
```
#### Frame prefix
When loading the same URDF multiple times (e.g. a dual-arm setup), use `frame_prefix` to give each instance unique frame IDs and `entity_path_prefix` to separate their geometry in the entity tree. Use `log_urdf_to_recording()` to log the model with prefixed frame IDs:
```python
left = rr.urdf.UrdfTree.from_file_path("robot.urdf", entity_path_prefix="left", frame_prefix="left/")
right = rr.urdf.UrdfTree.from_file_path("robot.urdf", entity_path_prefix="right", frame_prefix="right/")
left.log_urdf_to_recording()
right.log_urdf_to_recording()
```
Transforms computed via `joint.compute_transform()` will automatically use the prefixed frame IDs (e.g. `"left/base"`, `"right/shoulder"`).
### UrdfJoint
Each joint exposes properties from the URDF file:
* `name`
* `joint_type` (e.g. `revolute`, `continuous`, `prismatic`, `fixed`)
* `parent_link`, `child_link`
* `axis`, `origin_xyz`, `origin_rpy`
* `limit_lower`, `limit_upper`, `limit_effort`, `limit_velocity`
Use `compute_transform()` to get a [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md) with the correct `parent_frame` and `child_frame` already set:
```python
# For revolute/continuous joints: pass angle in radians
# For prismatic joints: pass distance in meters
transform = joint.compute_transform(angle)
rec.log("transforms", transform)
```
## Load URDF into an existing recording
If you already have a recording with transforms loaded in Rerun and want to add an URDF to it, you can do so via drag-and-drop or the menu ("Import into current recording").
In this video, we load an ROS 2 `.mcap` file with TF messages that automatically get translated into Rerun [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md).
As indicated by the errors displayed in the viewer, there are some connections missing in the transform tree of this example MCAP.
In our case, these missing transforms are static links that are stored in URDF models separate from the MCAP file.
To add them, we can simply drag the corresponding URDF files into the viewer where we have loaded the MCAP:
## References
* [π Python `log_file_from_path`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log_file_from_path)
* [π¦ Rust `log_file_from_path`](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.log_file_from_path)
* [π C++ `log_file_from_path`](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a20798d7ea74cce5c8174e5cacd0a2c47)
# Send partial updates over time
Rerun allows you to log only the data that has changed in-between frames (or whatever atomic unit [your timeline](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) is using), i.e. you can think of this as a sort of diffs or delta encodings.
This is a natural consequence of how Rerun [ingests, stores](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) and finally [queries](https://rerun.io/docs/concepts/visualization/entity-queries.md) data: Rerun *always* operates that way, whether you're aware of it or not. Consider this simple snippet:
```python
"""Log some very simple points."""
import rerun as rr
rr.init("rerun_example_points3d", spawn=True)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]]))
```
Here, only the positions of the points have been specified but, looking at the [complete definition for Points3D](https://rerun.io/docs/reference/types/archetypes/points3d.md), we can see that it has quite a few more [components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) available:
> **Required**: [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md)
>
> **Recommended** & **Optional**: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md), [`Color`](https://rerun.io/docs/reference/types/components/color.md), [`Text`](https://rerun.io/docs/reference/types/components/text.md), [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md), [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md), [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md)
All three languages for which we provide logging SDKs (Python, Rust, C++) expose APIs that allow fine-grained control over which components of an archetypes, when, and how.
The best way to learn about these APIs is to see them in action: check out the examples below.
## Examples
### Update specific properties of a point cloud over time
```python
"""Update specific properties of a point cloud over time."""
import rerun as rr
rr.init("rerun_example_points3d_partial_updates", spawn=True)
positions = [[i, 0, 0] for i in range(10)]
rr.set_time("frame", sequence=0)
rr.log("points", rr.Points3D(positions))
for i in range(10):
colors = [[20, 200, 20] if n < i else [200, 20, 20] for n in range(10)]
radii = [0.6 if n < i else 0.2 for n in range(10)]
# Update only the colors and radii, leaving everything else as-is.
rr.set_time("frame", sequence=i)
rr.log("points", rr.Points3D.from_fields(radii=radii, colors=colors))
# Update the positions and radii, and clear everything else in the process.
rr.set_time("frame", sequence=20)
rr.log(
"points",
rr.Points3D.from_fields(clear_unset=True, positions=positions, radii=0.3),
)
```
### Update specific properties of a transform over time
```python
"""Update specific properties of a transform over time."""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_partial_updates", spawn=True)
# Set up a 3D box.
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
)
# Update only the rotation of the box.
for deg in range(46):
rad = truncated_radians(deg * 4)
rr.log(
"box",
rr.Transform3D.from_fields(
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=rad
),
),
)
# Update only the position of the box.
for t in range(51):
rr.log(
"box",
rr.Transform3D.from_fields(translation=[0, 0, t / 10.0]),
)
# Update only the rotation of the box.
for deg in range(46):
rad = truncated_radians((deg + 45) * 4)
rr.log(
"box",
rr.Transform3D.from_fields(
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=rad
),
),
)
# Clear all of the box's attributes.
rr.log(
"box",
rr.Transform3D.from_fields(clear_unset=True),
)
```
### Update specific parts of a 3D mesh over time
```python
"""Log a colored triangle, then update its vertices' positions each frame."""
import numpy as np
import rerun as rr
rr.init("rerun_example_mesh3d_partial_updates", spawn=True)
vertex_positions = np.array(
[[-1.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], dtype=np.float32
)
# Log the initial state of our triangle
rr.set_time("frame", sequence=0)
rr.log(
"triangle",
rr.Mesh3D(
vertex_positions=vertex_positions,
vertex_normals=[0.0, 0.0, 1.0],
vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255]],
),
)
# Only update its vertices' positions each frame:
for i in range(1, 300):
factor = np.abs(np.sin(i * 0.04))
rr.set_time("frame", sequence=i)
rr.log(
"triangle",
rr.Mesh3D.from_fields(vertex_positions=vertex_positions * factor),
)
```
# How does Rerun work?
Rerun has several components manage multimodal data across its lifetime. This page explains what they are and how they connect.
## The components
### Logging SDK
The Logging SDK is how you get data into Rerun. Available for Python, Rust, and C++, it runs inside your application and logs data using [archetypes](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) β structured types like `Points3D`, `Image`, or `Transform3D`.
Data can be streamed directly to the Viewer, saved to `.rrd` files, or both.
### Viewer
The Viewer visualizes your data. It comes in two forms:
- **Native Viewer**: A desktop application for Linux, macOS, and Windows
- **Web Viewer**: A browser based application
The viewer includes a [**Chunk Store**](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) (in-memory database for logged data) and a **gRPC endpoint** that accepts streamed data from the SDK.
The Web Viewer has performance limitations compared to the native viewer. It runs as 32-bit Wasm and is limited to ~2 GiB memory in practice, limiting the amount of data that can be visualized simultaneously. It also runs single-threaded, making it generally slower than native.
Both viewers can be extended: the Native Viewer through its [Rust API](https://rerun.io/docs/howto/visualization/extend-ui.md), and the Web Viewer can be [embedded in web applications](https://rerun.io/docs/howto/integrations/embed-web.md) or [Jupyter notebooks](https://rerun.io/docs/howto/integrations/embed-notebooks.md).
### Catalog server
The catalog server provides persistent storage and indexing for large-scale data. It organizes data into:
- **Datasets**: Named collections of related recordings
- **Segments**: Individual `.rrd` files registered to a dataset
Data is served via the **redap** protocol (**Re**run **Da**ta **P**rotocol).
The catalog server is available as:
- Open-source server for local development (`rerun server`)
- **Rerun Hub**, our managed offering for production deployments
### Catalog SDK
The Catalog SDK (`rerun.catalog`) is a Python library for querying and manipulating the data stored on a catalog server. Combined with Rerun Hub, it allows building complex data transformation pipelines.
## How they connect
## What ships where?
### Hosted web viewer
The Web Viewer is available at [rerun.io/viewer](https://rerun.io/viewer).
It's a great place to start exploring the examples.
### CLI
The `rerun` binary bundles multiple tools in one:
- **Native Viewer** for visualization
- **OSS catalog server** (via `rerun server`)
- **RRD tools** for file manipulation
- **Web Viewer** (via `rerun --serve-web`)
The Rerun CLI can be downloaded from [GitHub](https://github.com/rerun-io/rerun/releases) or as part of the Python SDK.
It can also be built from source with `cargo install rerun-cli --locked`.
See: [CLI reference](https://rerun.io/docs/reference/cli.md)
### Python SDK
The Python SDK includes:
- **Logging SDK**
- **Catalog SDK**
- **CLI**, including the Viewer (the `rerun` CLI is made available by installing the `rerun-sdk` Python package)
See: Python SDK [installation instructions](https://rerun.io/docs/getting-started/install-rerun/python.md) and [quick start guide](https://rerun.io/docs/getting-started/data-in.md)
### Rust SDK
The Logging SDK as a Rust crate.
See: Rust SDK [installation instructions](https://rerun.io/docs/getting-started/install-rerun/rust.md) and [quick start guide](https://rerun.io/docs/getting-started/data-in.md)
### C++ SDK
The Logging SDK for C++ projects.
See: C++ SDK [installation instructions](https://rerun.io/docs/getting-started/install-rerun/cpp.md) and [quick start guide](https://rerun.io/docs/getting-started/data-in.md)
### The `web-viewer` and `web-viewer-react` NPM packages
These NPM packages bundle the Web Viewer for inclusion on a website.
See: the `web-viewer` package [reference](https://rerun.io/docs/reference/npm.md)
## Common workflows
### Stream to Viewer
The simplest workflow: stream data directly from your code to the Viewer for live visualization.
Minimal example:
```python
import rerun as rr
rr.init("rerun_example_log_to_grpc")
# Connect to the Rerun gRPC server using the default address and url: rerun+http://localhost:9876/proxy
rr.connect_grpc()
# Log data as usual, thereby pushing it into the gRPC connection.
while True:
rr.log("/", rr.TextLog("Logging thingsβ¦"))
```
Best for: development, debugging, real-time monitoring.
### Save to RRD, view later
Log data to `.rrd` files, then open them in the Viewer whenever needed. Files can be loaded from disk or URLs.
Minimal example:
```python
import rerun as rr
rr.init("rerun_example_log_to_rrd")
# Open a local file handle to stream the data into.
rr.save("/tmp/my_recording.rrd")
# Log data as usual, thereby writing it into the file.
while True:
rr.log("/", rr.TextLog("Logging thingsβ¦"))
```
And later:
```sh
$ rerun /tmp/my_recording.rrd
```
Best for: sharing recordings, offline analysis, archiving.
### Store on a catalog server
Register `.rrd` files with a catalog server for persistent, indexed storage. Query and visualize on demand.
Minimal example of creating a dataset and registering files:
```python
import rerun as rr
client = rr.catalog.CatalogClient("rerun://example.cloud.rerun.io")
dataset = client.create_dataset("my_data")
dataset.register(["s3://my-rrd-files/recording1.rrd", "s3://my-rrd-files/recording2.rrd"])
```
Best for: large datasets, team collaboration, production pipelines.
### Query and transform data
Use the Catalog SDK to query data from a catalog server, process it, and write results back. Visualization is available at any time.
Minimal example of querying a dataset:
```python
import datafusion as dfn
import rerun as rr
client = rr.catalog.CatalogClient("rerun://example.cloud.rerun.io")
dataset = client.get_dataset("my_data")
df = dataset.filter_contents("/obs").reader(index="log_time") # `df` is a DataFusion dataframe
df.filter(dfn.col("obs:Scalars:scalars").is_not_null()).count() # count observations in recording
```
Best for: data pipelines, batch processing, ML training data preparation.
# Log and Ingest
This section covers how data is structured and ingested into Rerun.
- [Recordings](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) - managing recordings and application IDs
- [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) - Rerun's data model
- [The Entity Path Hierarchy](https://rerun.io/docs/concepts/logging-and-ingestion/entity-path.md) - organizing data hierarchically
- [Transforms & Coordinate Frames](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md) - working with coordinate systems
- [Events and Timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) - managing temporal data
- [Static data](https://rerun.io/docs/concepts/logging-and-ingestion/static.md) - data that exists across all timelines
- [Chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) - internal storage mechanism
- [Importers](https://rerun.io/docs/concepts/logging-and-ingestion/importers.md) - loading external file formats
- [Query semantics & partial updates](https://rerun.io/docs/concepts/logging-and-ingestion/latest-at.md) - how Rerun resolves data queries
- [MCAP files](https://rerun.io/docs/concepts/logging-and-ingestion/mcap.md) - working with MCAP files
- [Component Batches](https://rerun.io/docs/concepts/logging-and-ingestion/batches.md) - efficiently logging collections of data
- [Sinks](https://rerun.io/docs/concepts/logging-and-ingestion/sinks.md) - where logged data goes
- [Video](https://rerun.io/docs/concepts/logging-and-ingestion/video.md) - video data support
# Train
A Rerun [catalog](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) can feed training pipelines two ways: export recordings to a standard format, or stream them directly into a PyTorch `DataLoader`.
## Export to a training format
The catalog exposes recordings as queryable DataFrames via [DataFusion](https://datafusion.apache.org/python/).
Multi-rate sensor streams can be time-aligned and columns of interest extracted, with the result written to whatever format a training pipeline expects.
See [Export recordings to LeRobot datasets](https://rerun.io/docs/howto/train/lerobot_export.md) for a worked example.
## Train directly from the catalog
The experimental [`rerun.experimental.dataloader`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/) module wraps a catalog as iterable or map-style PyTorch datasets, with no intermediate export step.
### Sample space
Three things describe a dataset (see [reference](https://ref.rerun.io/docs/python/stable/experimental_dataloader/)):
- **[`DataSource`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.DataSource)** β a catalog `DatasetEntry` with an optional segment filter; each registered RRD is one *segment*, typically one episode or trajectory
- **`index`** β the timeline that defines what "one sample" means (e.g. `"frame_index"` or `"real_time"`)
- **`fields`** β a dict of [`Field`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.Field)s, each mapping a source column (an `entity:Archetype:component` triple) to a decoder
[`SampleIndex`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.SampleIndex) pre-computes the full sample space from lightweight per-segment index-range metadata β one query per segment, not a scan of the data.
For timestamp timelines, `FixedRateSampling` defines the sampling grid and the server handles drift between grid and real row positions via `fill_latest_at`.
### Decoders
Each `Field` has a `ColumnDecoder` ([`_decoders.py`](https://github.com/rerun-io/rerun/blob/main/rerun_py/rerun_sdk/rerun/experimental/dataloader/_decoders.py)) that converts a raw Arrow column to a `torch.Tensor`:
- [`NumericDecoder`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.NumericDecoder) β scalars and numeric lists
- [`ImageDecoder`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.ImageDecoder) β JPEG/PNG blobs
- [`VideoFrameDecoder`](https://ref.rerun.io/docs/python/stable/experimental_dataloader/#rerun.experimental.dataloader.VideoFrameDecoder) β compressed video (`h264`/`h265`/`av1`)
### Windows
`Field(window=(start, end))` returns a slice of values across an inclusive range relative to the current sample rather than a single value.
This is how action chunks and observation history are expressed.
### Dataset styles
- `RerunIterableDataset` β streaming with automatic shuffling and cross-worker and DDP partitioning
- `RerunMapDataset` β random access by global index; works with PyTorch samplers like `DistributedSampler` and `WeightedRandomSampler`
See [Train PyTorch models with Rerun](https://rerun.io/docs/howto/train/dataloader.md) for usage.
# Query and Transform
This section will cover how to query and transform data in Rerun.
# Visualize
This section covers how data is visualized in the Rerun Viewer.
- [Blueprints](https://rerun.io/docs/concepts/visualization/blueprints.md) - configuring visualization layouts and views
- [Customize views](https://rerun.io/docs/concepts/visualization/customize-views.md) - visualizers, overrides, and per-entity customization
- [Annotation Context](https://rerun.io/docs/concepts/visualization/annotation-context.md) - shared styling and labels
- [Entity Queries](https://rerun.io/docs/concepts/visualization/entity-queries.md) - controlling which entities appear in a view
# Customize views
This section explains the process by which logged data is used to produce a visualization and how it can be customized via the user interface or code.
## How are visualizations produced?
In the Rerun Viewer, visualizations happen within _views_, which are defined by their [_blueprint_](https://rerun.io/docs/concepts/visualization/blueprints.md).
The first step for a view to display its content is to determine which entities are involved.
This is determined by the [entity query](https://rerun.io/docs/concepts/visualization/entity-queries.md), which is part of the view blueprint.
The query is run against the data store to generate the list of view entities.
Views rely on visualizers to display each of their entities.
For example, [3D views](https://rerun.io/docs/reference/types/views/spatial3d_view.md) use the `Points3D` visualizer to display 3D point clouds,
and [time series views](https://rerun.io/docs/reference/types/views/time_series_view.md) use the `SeriesLines` visualizer to display time series line plots.
Which visualizers are available is highly dependent on the specific kind of view.
For example, the `SeriesLines` visualizer only exists for time series views β not, e.g., for 3D views.
For a given view, each entity's components determine which visualizers are available.
By default, visualizers are selected for entities logged with a corresponding [archetype](https://rerun.io/docs/reference/types/archetypes.md).
For example, in a 3D view, an entity logged with the [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md) archetype results in the `Points3D` visualizer being selected by default.
This happens because the components of an [archetype](https://rerun.io/docs/reference/types/archetypes.md) are tagged with the archetype's name.
With a few exceptions, archetypes are directly associated with a single visualizer, but it's also possible to add multiple visualizers of the same type to a given entity via blueprints or the UI.
Then, each selected visualizer determines the values for the components it supports. For example, the `Points3D` visualizer handles, among others, the [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md), [`Radius`](https://rerun.io/docs/reference/types/components/radius.md), and [`Color`](https://rerun.io/docs/reference/types/components/color.md) components.
Sometimes it makes sense to explicitly set the visualizers, to change the way entities are visualized.
Here is how to force a `SeriesPoints` visualizer for `/trig/sin`, in addition to the default `SeriesLines` visualizer:
```python
"""Log a scalar over time and override the visualizer."""
from math import cos, sin, tau
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_series_line_overrides", spawn=True)
# Log the data on a timeline called "step".
for t in range(int(tau * 2 * 10.0)):
rr.set_time("step", sequence=t)
rr.log("trig/sin", rr.Scalars(sin(float(t) / 10.0)))
rr.log("trig/cos", rr.Scalars(cos(float(t) / 10.0)))
# Use the SeriesPoints visualizer for the sin series.
rr.send_blueprint(
rrb.TimeSeriesView(
overrides={
"trig/sin": [rr.SeriesLines(), rr.SeriesPoints()],
},
),
)
```.py
The view now displays a series of points in addition to connecting the values with lines.
Here is how the visualizers are displayed in the user interface:
The next section describes how to precisely control what data each visualizer operates on, to fully customize the contents of a view.
## Component mappings
Each visualizer takes various components as input. Values are automatically sourced from the data store. When no matching data exists (except for required components like point cloud positions or plot scalars), the Viewer generates sensible default values.
The exact way this is done depends on the type of View, but may be influenced by a variety of circumstances.
Component mappings let you customize this behavior, for example to:
* Control what data is picked from the store - this allows you to visualize arbitrary data, _even when it was not logged with Rerun-semantics_.
* Specify the styling of a visualization as part of your blueprint
Component mappings can be modified via the Viewer UI by navigating to a visualizer and expanding the component of interest.
### Custom values
A common way of customizing a visualization is by setting custom values, for example for visualizers that expect a [`Color`](https://rerun.io/docs/reference/types/components/color.md) component.
In the UI this can be done via the visualizer UI, by clicking and modifying the color component, or by selecting "Add customβ¦" from the Source dropdown.
When such a customization is defined, it automatically changes the component's source for this visualizer to point to this new custom value.
The Source dropdown menu allows quick toggling between the different input representations.
By clicking on "Add customβ¦" you can create a new custom component override:
You then can use the color picker to determine a color:
Note that any direct edit on any component of the visualizer will always set the source to "Custom".
The following snippet shows how the same customization can be achieved with the blueprint API:
```python
"""Override a component."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_component_override", spawn=True)
# Data logged to the data store.
rr.log("boxes/1", rr.Boxes2D(centers=[0, 0], sizes=[1, 1], colors=[255, 0, 0]))
rr.log("boxes/2", rr.Boxes2D(centers=[2, 0], sizes=[1, 1], colors=[255, 0, 0]))
rr.send_blueprint(
rrb.Spatial2DView(
# Override the values from the data store for the first box.
overrides={
"boxes/1": rr.Boxes2D(colors=[0, 255, 0]),
},
),
)
```
### Remapping of components
A powerful mechanism that is built into visualizers is the option to source components from data that was logged on the same entity but might have arbitrary semantics.
Within a view, a visualizer can pick up any component that has the same datatype as the builtin type that it expects.
For example, the `SeriesLines` and `SeriesPoints` visualizers can pick up any numerical data for their [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md) component.
The same holds for String-like components that can be selected for [`Name`](https://rerun.io/docs/reference/types/components/name.md).
Likewise, the state timeline view's visualizer can source its `StateChange:state` input from any string, boolean, or numeric component (see [Visualize state changes](https://rerun.io/docs/howto/visualization/state-timeline.md)).
Such data often comes from MCAP data that has user-defined message types, or from components that were flexibly logged via [`AnyValues`](https://ref.rerun.io/docs/python/main/common/custom_data/#rerun.AnyValues) or [`DynamicArchetype`](https://ref.rerun.io/docs/python/main/common/custom_data/#rerun.DynamicArchetype).
The Viewer can even look for data with compatible datatypes in nested fields of Arrow [`StructArrays`](https://docs.rs/arrow/latest/arrow/array/struct.StructArray.html).
Suitable components show up in the source dropdown:
> #12661: Currently, only the time series view (scalars) and the state timeline view (state values) allow remapping of required components. All other visualizers require matching Rerun semantics (correct archetype & type metadata) for their required fields.
As always, component mappings can be set via the blueprint APIs:
```python
# Green cosine:
# * source scalars from the custom component
# "custom:my_custom_scalar"
# * set the name via an override
# * everything else uses the automatic component mappings,
# so it will pick up colors from the view default.
rr.SeriesLines(names="cosine (custom)").visualizer(
mappings=[
# Map scalars to the custom component.
VisualizerComponentMapping(
target="Scalars:scalars",
source_kind=ComponentSourceKind.SourceComponent,
# Map from custom component
source_component="custom:my_custom_scalar",
),
]
),
```
To select nested fields in `StructArrays`, Rerun uses so-called selectors, which are filters that are inspired by [`jq`](https://jqlang.org/), a tool for processing JSON data.
## Per-view component default
The viewer picks default component values based on a wide variety of different heuristics, ranging from simple local properties like
an entity's name all the way to things like the size of a view's bounding box or how many plots are within it.
Sometimes, it makes sense to set a _custom_ default value that is applied across all
visualizers of a view, to avoid redundant blueprint definitions.
This can be done through the UI by selecting the view in question and specifying a new default there:
A custom default value set this way will show up on the respective visualizers:
This is how it is achieved with the blueprint API:
```python
"""Add a component default."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_component_override", spawn=True)
# Data logged to the data store.
rr.log("boxes/1", rr.Boxes2D(centers=[0, 0], sizes=[1, 1], colors=[255, 0, 0]))
rr.log("boxes/2", rr.Boxes2D(centers=[2, 0], sizes=[1, 1]))
rr.send_blueprint(
rrb.Spatial2DView(
overrides={"boxes/1": rr.Boxes2D(colors=[0, 255, 0])},
# Add a default value for all Color components in this view
defaults=[rr.Boxes2D.from_fields(colors=[0, 0, 255])],
),
)
```
Here, the `/boxes/2` entity is no longer logged with a color value, but a default box color is added to the blueprint. Here is how the user interface represents its visualizer:
And as before, this also shows up in the View's component defaults.
# Entity Queries
Many views are made up of visualizations that include more than one
entity.
Rather that requiring you to specify each entity individually, Rerun supports
this through "entity queries" that allow you to use "query expressions" to
include or exclude entire subtrees.
## Query expression syntax
An entity query is made up of a set of "query expressions." Each query expression
is either an "inclusion," which starts with an optional `+` or an "exclusion,"
which always starts with a `-`.
Query expressions are also allowed to end with an optional `/**`. The`/**`
suffix matches the whole subtree, i.e. self and any child, recursively. For
example, `/world/**`matches both`/world`and`/world/car/driver`. Other uses of
`*` are not yet supported.
When combining multiple query expressions, the rules are sorted by entity-path,
from least to most specific:
- If there are multiple matching rules, the most specific rule wins.
- If there are multiple rules of the same specificity, the last one wins.
- If no rules match, the path is excluded.
Consider the following example:
```diff
+ /world/**
- /world
- /world/car/**
+ /world/car/driver
```
- The last rule matching `/world/car/driver` is `+ /world/car/driver`, so it
is included.
- The last rule matching `/world/car/hood` is `- /world/car/**`, so it is
excluded.
- The last rule matching `/world` is `- /world`, so it is excluded.
- The last rule matching `/world/house` is `+ /world/**`, so it is included.
## In the Viewer
In the viewer, an entity query is typically displayed as a multi-line
edit box, with each query expression shown on its own line. You can find the
query editor in the right-hand selection panel when selecting a view.
## In the SDK
In the SDK, query expressions are represented as a list or iterable, with each
expression written as a separate string. The query expression from above would
be written as:
```python
(
rrb.Spatial3DView(
contents=[
"+ helix/**",
"- helix/structure/scaffolding",
],
),
)
```
## `origin` substitution
Query expressions also allow you to use the variable `$origin` to refer to the
origin of the view that the query belongs to.
For example, the above query could be rewritten as:
```python
(
rrb.Spatial3DView(
origin="helix",
contents=[
"+ $origin/**",
"- $origin/structure/scaffolding",
],
),
)
```
# Annotation Context
## Overview
Any visualization that assigns an identifier ("Class ID") to an instance or entity can benefit from using Annotations.
By using an Annotation Context, you can associate labels and colors with a given class and then re-use
that class across entities.
This is particularly useful for visualizing the output of classifications algorithms
(as demonstrated by the [Detect and Track Objects](https://github.com/rerun-io/rerun/tree/main/examples/python/detect_and_track_objects) example),
but can be used more generally for any kind of reoccurring categorization within a Rerun recording.
### Keypoints & keypoint connections
Rerun allows you to define keypoints *within* a class.
Each keypoint can define its own properties (colors, labels, etc.) that overwrite its parent class.
A typical example usage of keypoints is annotating the joints of a skeleton within a pose detection.
In that case, the entire detected pose/skeleton is assigned a Class ID and each joint within gets a Keypoint ID.
To help you more with this (and similar) use-case(s), you can also define connections between keypoints
as part of your annotation class description.
The Viewer will draw the connecting lines for all connected keypoints whenever that class is used.
Just as with labels and colors this allows you to use the same connection information on any instance that class in your scene.
Keypoints are currently only applicable to 2D and 3D points.
### Logging an annotation context
Annotation Context is typically logged as [static](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) data, but can change over time if needed.
The Annotation Context is defined as a list of Class Descriptions that define how classes are styled
(as well as optional keypoint style and connection).
Annotation contexts are logged with:
* Python: π[`rr.AnnotationContext`](https://ref.rerun.io/docs/python/stable/common/archetypes/#rerun.archetypes.AnnotationContext)
* Rust: π¦[`rerun::AnnotationContext`](https://docs.rs/rerun/latest/rerun/archetypes/struct.AnnotationContext.html#)
```python
import rerun as rr
rr.init("rerun_example_annotation_context_connections")
# Annotation context with two classes, using two labeled classes, of which
# ones defines a color.
rr.log(
"masks", # Applies to all entities below "masks".
rr.AnnotationContext(
[
rr.AnnotationInfo(id=0, label="Background"),
rr.AnnotationInfo(id=1, label="Person", color=(255, 0, 0)),
],
),
static=True,
)
# Annotation context with simple keypoints & keypoint connections.
rr.log(
"detections", # Applies to all entities below "detections".
rr.ClassDescription(
info=rr.AnnotationInfo(0, label="Snake"),
keypoint_annotations=[
rr.AnnotationInfo(id=i, color=(0, 28 * i, 0)) for i in range(10)
],
keypoint_connections=[(i, i + 1) for i in range(9)],
),
static=True,
)
```
## Affected entities
Each entity that uses a Class ID component (and optionally Keypoint ID components) will look for
the nearest ancestor that in the [entity path hierarchy](https://rerun.io/docs/concepts/logging-and-ingestion/entity-path.md) that has an Annotation Context defined.
## Segmentation images
Segmentation images are single channel integer images/tensors where each pixel represents a class id.
By default, Rerun will automatically assign colors to each class id, but by defining an Annotation Context,
you can explicitly determine the color of each class.
* Python: [`rr.SegmentationImage`](https://ref.rerun.io/docs/python/stable/common/archetypes/#rerun.archetypes.SegmentationImage)
* Rust: Log a [`rerun::SegmentationImage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.SegmentationImage.html)
# Blueprints
## What are Blueprints?
When you work with the Rerun Viewer, understanding blueprints is important if you want to build consistency around your Viewer experience.
*For a video overview, check out the [Blueprints video](https://www.youtube.com/embed/kxbkbFVAsBo?si=k2JPz3RbhR1--pcw) on YouTube.*
VIDEO
A way to think about the Rerun View is that
- The **recording** provides the actual data you are visualizing
- The **blueprint** determines how that data is displayed
Both pieces are crucial. Without a recording there is nothing to show. Without a blueprint there is no way to show it. Even when you use Rerun without explicitly loading a blueprint, the Viewer creates one automatically for you.
## What blueprints control
Blueprints give you complete control over the Viewer's layout and configuration:
- **Panel visibility**: Whether panels like the blueprint panel, selection panel, and time panel are expanded or collapsed
- **Layout structure**: How views are arranged using containers (Grid, Horizontal, Vertical, Tabs)
- **View types and configuration**: What kind of views display your data (2D/3D spatial, maps, charts, text logs, etc.) and their specific settings
- **Visual properties**: Styling like backgrounds, colors, zoom levels, time ranges, and visual bounds
In general, if you can modify an aspect of how something looks through the Viewer, you are actually modifying the blueprint.
## Application IDs: binding blueprints to data
The [Application ID](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) is how blueprints connect to your data. This is a critical concept:
**All recordings that share the same Application ID will use the same blueprint.**
This loose coupling between blueprints and recordings means:
- You can keep the blueprint constant while changing the recording to compare different datasets with consistent views
- You can change the blueprint while keeping a recording constant to view the same data in different ways
- When you save blueprint changes with the Viewer, those changes apply to all recordings with that Application ID
Think of the Application ID as the "key" that binds a blueprint to a specific type of recording. If you want recordings to share the same layout, give them the same Application ID.
## Reset behavior: heuristic vs default
The Viewer provides two types of blueprint reset, accessible from the blueprint panel:
### Reset to heuristic blueprint
This generates a new blueprint automatically based on your current data. The Viewer analyzes what you've logged and creates an appropriate layout using built-in heuristics. This is useful when you want to start fresh and let Rerun figure out a reasonable layout.
### Reset to default blueprint
This returns to your programmatically specified blueprint (sent from code) or a saved blueprint file (`.rbl`). If you've sent a blueprint using `rr.send_blueprint()` or loaded a `.rbl` file, this becomes your "default." The reset button in the blueprint panel will restore this default whenever you need it.
When no default blueprint has been set, the reset button will use the heuristic blueprint instead.
## Three ways to work with blueprints
There are three complementary approaches to creating and modifying blueprints:
### 1. Interactively
Modify blueprints directly in the Viewer UI:
- Drag and drop views to rearrange them
- Add new views or containers with the "+" button
- Split views horizontally, vertically, or into grids
- Change container types (Grid, Horizontal, Vertical, Tabs)
- Rename views and containers
- Show, hide, or remove elements
This is the fastest way to experiment with layouts. See [Configure the Viewer](https://rerun.io/docs/getting-started/configure-the-viewer.md) for a complete guide.
### 2. Save and load files
Save your blueprint configuration to `.rbl` files:
- Use "Save blueprintβ¦" from the file menu to save your current layout
- Load blueprints with "Openβ¦" or by dragging `.rbl` files into the Viewer
- Share blueprint files with teammates to ensure everyone sees data the same way
- Reuse blueprints across sessions and different recordings (with the same Application ID)
Blueprint files are portable and can be version-controlled alongside your code.
### 3. Programmatically
Write blueprint code that configures the Viewer automatically:
- Define layouts in Python using `rerun.blueprint` APIs
- Send blueprints with `rr.send_blueprint()` or via `default_blueprint` parameter
- Generate layouts dynamically based on your data
- Perfect for creating consistent views for specific debugging scenarios
For example, you might send different blueprints automatically based on detected issues in your application (e.g., a robot enters an error state and surfaces the correct blueprint to help you debug that)
```python
import rerun as rr
import rerun.blueprint as rrb
if robot_error:
# Show diagnostic views for debugging
blueprint = rrb.Grid(
rrb.Spatial3DView(name="Robot view", origin="/world/robot"),
rrb.TextLogView(name="Error Logs", origin="/diagnostics"),
rrb.TimeSeriesView(name="Sensor Data", origin="/sensors"),
)
rr.send_blueprint(blueprint, make_active=True)
```
See [Configure the Viewer](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md) for detailed examples and our guide on how to [build a blueprint programmatically](https://rerun.io/docs/howto/visualization/build-a-blueprint-programmatically.md).
## Common use cases
### Debugging specific scenarios
Create blueprints optimized for diagnosing particular issues. For example, when debugging robot perception, you might want a blueprint that shows:
- The camera view in 2D
- The 3D world with detected objects
- Detection confidence scores in a time series chart
- Error logs in a text panel
### Sharing layouts with teams
Save a blueprint file and share it with your team. Everyone loading that blueprint with matching recordings will see the data the same way, making it easier to discuss findings and collaborate.
### Templating for different data types
Create different blueprint templates for different types of recordings. For example:
- A blueprint for autonomous vehicle data that focuses on map views and sensor fusion
- A blueprint for robotics manipulation that emphasizes joint angles and gripper cameras
- A blueprint for computer vision that shows side-by-side comparisons of different models
### Dynamic Viewer configuration
Generate blueprints programmatically based on runtime conditions. For instance, automatically create one view per detected anomaly, or adjust the layout based on how many data sources are active.
## Blueprint architecture
Under the hood, blueprints are just data. They are structured using the same [Entity Component System](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) as your recordings, but with blueprint-specific archetypes and a separate blueprint timeline. This architecture provides several advantages:
- **Anything you modify in the Viewer can be saved and shared** as a blueprint file
- **Blueprints can be produced programmatically** using just the Rerun SDK without depending on the Viewer
- **Blueprint data is fully expressive**, enabling [blueprint overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) that are as powerful as logged data
- **The full time-series nature** simplifies future features like snapshots and undo/redo
- **Debugging tools for Rerun data** can inspect blueprint state just like recording data
### Viewer operation
The Viewer is designed to be deterministic. Every frame, the Viewer:
1. Takes the active blueprint and active recording
2. Queries container and view archetypes from the blueprint at the current blueprint timeline revision
3. Uses those view specifications to query the data needed from the recording
4. Renders the results
5. Queues any user interactions as new blueprint events on the blueprint timeline
This means the Viewer output is a deterministic function of the blueprint and the recording, with minimal persisted state between frames.
## Next steps
- **Learn to use blueprints**: See [Configure the Viewer](https://rerun.io/docs/getting-started/configure-the-viewer.md) for hands-on tutorials covering interactive, file-based, and programmatic workflows
- **Understand the UI**: Check the [Blueprint Panel Reference](https://rerun.io/docs/reference/viewer/blueprints.md) for details on UI controls
- **Customize visualizations**: Learn about [Visualizers and Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) for advanced per-entity customization
- **Explore the API**: Browse the [Blueprint API Reference](https://ref.rerun.io/docs/python/stable/common/blueprint_apis/) for programmatic control (Python)
# Lenses
> [!NOTE]
> The Lenses API is currently experimental and may change in future releases.
Lenses transform data by extracting, reshaping, and rerouting components.
They produce new component columns, entity paths, or timelines from existing data.
## Motivation
The goal of Rerun is to handle all kinds and shapes of user data.
In addition to the datatypes defined by Rerun, it is also possible to load data with user-defined types into the viewer and into `.rrd` files.
For example, [schema reflection](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md) can be used to import arbitrary Protobuf-based MCAP messages.
Using an expressive API, Lenses allow you to:
1. Reroute components to different entities
2. Attach Rerun semantics to arbitrary data
3. Wrangle the values stored in individual components
Lenses are available in the Rust SDK using `LensesSink` or directly on a `Chunk` via the `ChunkExt` trait.
In Python, Lenses can be applied to chunks directly or as a pipeline step in the `ChunkStream` API.
Internally, Rerun uses lenses to implement large parts of our data importers, the MCAP importer is one example of this.
## Example data
The examples below all operate on the same input chunk, logged to `/sensor/imu` with `frame` as a timeline and two component columns `Imu:accel` and `Imu:status`:
```python
# Build a chunk with a struct-typed component.
imu_data = pa.StructArray.from_arrays(
[
pa.array([1.0, 2.0, 3.0], type=pa.float64()),
pa.array([4.0, 5.0, 6.0], type=pa.float64()),
pa.array([0, 10_000_000, 20_000_000], type=pa.int64()),
],
names=["x", "y", "elapsed"],
)
status_data = pa.array(["ok", "ok", "warn"], type=pa.utf8())
chunk = Chunk.from_columns(
"/sensor/imu",
indexes=[rr.TimeColumn("frame", sequence=[0, 1, 2])],
columns=rr.DynamicArchetype.columns(
archetype="Imu", components={"accel": imu_data, "status": status_data}
),
)
```
| `frame` | `Imu:accel` | `Imu:status` |
|------:|-----------|------------|
| 0 | `[{x: 1.0, y: 4.0, elapsed: 0}]` | `["ok"]` |
| 1 | `[{x: 2.0, y: 5.0, elapsed: 10000000}]` | `["ok"]` |
| 2 | `[{x: 3.0, y: 6.0, elapsed: 20000000}]` | `["warn"]` |
## Derive lenses
A derive lens creates **new** component columns from an input component.
It selects an input column, extracts data using a `Selector`, and writes the results as new columns (optionally at a different entity and with additional timelines).
The following lens extracts the `.y` field from the struct as a [`Scalar`](https://rerun.io/docs/reference/types/archetypes/scalars.md), extracts the `.elapsed` field as a new timeline, and writes both to the entity `/new_entity/accel_y`:
```python
# Extract the "y" field to a different entity and the "elapsed" field as a
# new timeline.
extract_y = (
DeriveLens("Imu:accel", output_entity="/new_entity/accel_y")
.to_component(rr.Scalars.descriptor_scalars(), ".y")
.to_timeline("sensor_elapsed", "duration_ns", ".elapsed")
)
```
See the full examples in [Rust](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/concepts/lenses.rs) and [Python](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/concepts/lenses.py).
When we apply the `extract_y` lens, we get the following resulting chunks.
On `/sensor/imu`, the unmodified `Imu:status` column remains:
| `frame` | `Imu:status` |
|------:|------------|
| 0 | `["ok"]` |
| 1 | `["ok"]` |
| 2 | `["warn"]` |
On `/new_entity/accel_y`, we get the extracted [`Scalar`](https://rerun.io/docs/reference/types/archetypes/scalars.md) column and the new `sensor_elapsed` timeline:
| `frame` | `sensor_elapsed` | `Scalars:scalars` |
|------:|------:|-----------|
| 0 | 0 | `[4.0]` |
| 1 | 10000000 | `[5.0]` |
| 2 | 20000000 | `[6.0]` |
Note that the original `frame` timeline is present for all entities with the correct values.
## Mutate lenses
A mutate lens modifies an existing component column by applying a selector to it.
Unlike derive lenses, no new columns are created.
The input column is transformed and stays at the same entity.
The following lens simplifies the `Imu:accel` struct to just its `.x` field:
```python
# Simplify the accel struct to just its "x" field in-place.
simplify_accel = MutateLens("Imu:accel", ".x")
```
After applying the `simplify_accel` lens, `/sensor/imu` looks like this:
| `frame` | `Imu:accel` | `Imu:status` |
|------:|-----------|------------|
| 0 | `[1.0]` | `["ok"]` |
| 1 | `[2.0]` | `["ok"]` |
| 2 | `[3.0]` | `["warn"]` |
The struct has been replaced by the extracted float values, while `Imu:status` remains unchanged.
## Output modes
When streaming data through lenses, the output mode controls which components are forwarded:
* `ForwardUnmatched` forwards original components that are not consumed by any lens, alongside any lens-produced outputs.
* `ForwardAll` forwards all original components alongside lens-produced outputs. This leads to data duplication but can be helpful for debugging.
* `DropUnmatched` only forwards lens-produced outputs, dropping all other components.
## Selectors
The actual transformations of the contents and values within a given column are expressed using `Selectors`, which are concise, declarative expressions that are inspired by [`jq`](https://jqlang.org/).
Because a lot of user-defined types are hierarchically nested message definitions, this yields a natural way to describe extractions.
The basic syntax elements are:
* `.` - identity, selects the current value
* `.field` - access a named field (e.g. `.my.nested.struct.field`)
* `.sequence[]` - iterate over all elements in a sequence
* `.sequence[].x` - access a field on each element of a sequence
* `.optional_field?` - access an optional field, skipping missing values
* `pack(.x, .y, .z)` - pack several same-typed fields into a fixed-size list (see below)
These can be composed using pipes (`|`) as described below.
### Pipe
The `|` operator pipes the output of one expression into the next, just like a Unix pipe.
In the query string, this is useful for readability when chaining multiple steps: `.poses[] | .x`.
Beyond the query syntax, `Selector.pipe()` can also chain into arbitrary functions in the host language.
This is useful for value transformations that go beyond path navigation, like unit conversions or arithmetic.
For example, the following lens extracts the `.x` field and scales it by `9.81`:
```python
# Use pipe to apply a custom transformation after extracting a field.
extract_scaled_x = DeriveLens(
"Imu:accel", output_entity="/new_entity/accel_scaled_x"
).to_component(
rr.Scalars.descriptor_scalars(),
Selector(".x").pipe(lambda arr: pa.compute.multiply(arr, 9.81)),
)
```
### Packing fields into fixed-size lists
Many Rerun components are based on Arrow fixed-size lists.
For example, `Position3D` is a `FixedSizeList[3]`.
`pack(...)` assembles a fixed-size list from several paths that resolve to the same datatype, e.g. `pack(.x, .y, .z)`.
If a field is nullable, acknowledge it with `!` (for example `pack(.x!, .y!, .z!)`); a null in any field will null the corresponding row in the resulting array, potentially shadowing non-null data in other fields.
# Catalog object model
This page covers the catalog server's object model. For logging and recording basics, see [Recordings](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md). For API details, see the [Catalog SDK reference](https://ref.rerun.io/docs/python/stable/common/catalog/).
## Catalog
We refer to the contents stored in a given catalog server as the _catalog_.
The catalog contains top-level objects called _entries_.
There are currently two types of entries: **tables** and **datasets**.
Each is described in more detail below.
Entries share a few common properties:
- **id**: a globally unique identifier
- **name**: a user-provided name, which must be unique within the catalog
### Renaming a catalog entry
The id of a catalog entry is immutable, but the name can be changed provided it remains unique.
In Python, call `set_name()` on an entry to rename it on the catalog server, for example:
```python
client = rr.catalog.CatalogClient(β¦)
dataset = client.get_dataset("old_name")
dataset.set_name("new_name")
```
### Structuring datasets
When working with larger amounts of data, it can be useful to organize catalog entries in a directory-like structure.
This can be done by using `.` delimiters in the names.
The screenshot below is an example of a dot-delimited dataset name showing up as a directory tree in the viewer's data source browser:
## Table entries
Table entries model a single table of data.
They use the [Arrow data model](https://arrow.apache.org/docs/format/Columnar.html), so a table is logically equivalent to an [Arrow table](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html).
As a result, tables possess an [Arrow schema](https://arrow.apache.org/docs/python/generated/pyarrow.Schema.html).
Tables support the following mutation operations through the Catalog SDK:
- _append_: add new rows to the table
- _overwrite_: replace the entire table with new data
- _upsert_: replace existing rows (based on an index column) with new data
Thanks to [DataFusion](https://datafusion.apache.org/), tables also support most database operations such as querying, filtering, joining, etc.
## Datasets
Dataset entries model a collection of Rerun data organized in episodes such as recorded runs of a given robotic task.
These episodes within datasets are called _segments_, which are identified by a segment ID.
Segments are added to datasets by the process of _registering_ a [recording](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) (typically stored in some object store such as S3) to the dataset using the Catalog SDK.
The recording ID of the `.rrd` file is used as its segment ID.
Recordings registered to a given segment are organized by layers, identified by a layer name.
By default, the `"base"` layer name is used.
Registering two `.rrd` files with the same recording ID (that is, with the same segment ID) to the same dataset, and using the same layer name, will result in the second `.rrd` overwriting the first.
Additive registration can be achieved by using different layer names for different `.rrd`s with the same recording ID/segment ID.
Layers are immutable and can only be overwritten by registering a new `.rrd` file. In other words, datasets support the following mutation operations:
- _create segment_: by registering a `.rrd` with a "new" recording ID
- _append to segment_: by registering a `.rrd` with a matching recording ID to a new layer name
- _overwrite segment layer_: by registering a `.rrd` with a matching recording ID to an existing layer name
### Schema
Datasets are based on the Rerun data model, which consists of a collection of [chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) of Arrow data.
These chunks hold data for various [entities and components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) corresponding to various indexes (or [timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md)).
A given collection of chunks, say, a dataset segment, defines an Arrow schema.
We refer to this as _schema-on-read_, because the schema proceeds from the data, and not the other way around.
This differs from the table model, where the schema is defined upfront (_schema-on-write_).
In this context, the schema of a dataset is the union of schemas of its segments, which themselves are the union of the schemas of their layers.
Datasets maintain a minimal level of schema self-consistency.
Registering a `.rrd` whose schema is incompatible with the current dataset schema will result in an error.
In this context, _incompatible_ means that the schema of the new `.rrd` contains a column for the same entity, archetype, and component, but with a different Arrow type.
Such an occurrence is rare, and practically impossible when using standard Rerun archetypes.
### Blueprints
A dataset can be assigned a blueprint.
This is done by registering a `.rbl` blueprint file typically stored in object storage to the dataset.
A dedicated API exists for this in the Catalog SDK: [`DatasetEntry.register_blueprint()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry.register_blueprint).
In that case, the blueprint is applied to all segments of the dataset when visualized in the Rerun Viewer.
# Properties and segment tables
Properties are recording-level metadata that relates to an entire segment.
When you query a [dataset](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md), properties appear as columns in the segment table and can be used to filter, sort, and analyze your segments.
Common use cases for properties include tagging recordings with capture location, data format version, environmental conditions, or any other custom metadata relevant to your workflow.
## Understanding properties
Let's use an example to illustrate how properties work and how they can be retrieved and queried using a catalog server.
First, we create a few recordings with some properties:
```python
rrd_paths = [RRD_DIR / f"recording_{i}.rrd" for i in range(5)]
for i, rrd_path in enumerate(rrd_paths):
with rr.RecordingStream("rerun_example_property") as rec:
rec.save(rrd_path)
rec.log("data", rr.Points2D(positions=[[i, i]]))
# properties can be any rerun data
rec.send_property("location", rr.GeoPoints(lat_lon=[[46.5, 6.5]]))
# custom data can be logged with `AnyValues`
rec.send_property(
"info",
rr.AnyValues(
index=i,
is_odd=i % 2 == 1,
),
)
# recording name is part of the built-in properties
rr.send_recording_name(f"segment_{i}")
```
In this example, we use `send_property()` to attach metadata to each recording. Properties are regular Rerun data, so you can use any built-in archetype. Here we use [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md) to store a geographic location. For arbitrary data that doesn't fit an existing archetype, use [`AnyValues`](https://rerun.io/docs/howto/logging-and-ingestion/custom-data.md).
In addition to user-provided properties, Rerun automatically stores built-in properties using the [`RecordingInfo`](https://rerun.io/docs/reference/types/archetypes/recording_info.md) archetype.
Its `start_time` field is automatically populated, and its `name` field can be set with `send_recording_name()`.
Internally, properties are logged under a reserved `/__properties` entity path and use [static semantics](https://rerun.io/docs/concepts/logging-and-ingestion/static.md) since they apply to the entire recording rather than specific points in time.
## Querying the segment table
Once recordings are registered to a [dataset](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md), their properties become visible and queryable through the segment table.
Here we use the local open-source catalog server included with Rerun to illustrate this:
```python
# load the demo recording in a temporary catalog
with rr.server.Server(datasets={"dataset": rrd_paths}) as server:
# obtain a dataset from the catalog
dataset = server.client().get_dataset("dataset")
segment_table = dataset.segment_table()
# sort and select columns of interest
segment_table = segment_table.sort(
col("property:RecordingInfo:name")[0]
).select(
"rerun_segment_id",
"property:RecordingInfo:name",
"property:RecordingInfo:start_time",
"property:info:index",
"property:info:is_odd",
"property:location:GeoPoints:positions",
)
print(segment_table)
```
Output:
```
ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β rerun_segment_id β property:RecordingInfo:name β property:RecordingInfo:start_time β property:info:index β property:info:is_odd β property:location:GeoPoints:positions β
β --- β --- β --- β --- β --- β --- β
β type: Utf8 β type: nullable List[nullable Utf8] β type: nullable List[nullable i64] β type: nullable List[nullable i64] β type: nullable List[nullable bool] β type: nullable List[nullable FixedSizeList[nullable f64; 2]] β
β β archetype: RecordingInfo β archetype: RecordingInfo β component: index β component: is_odd β archetype: GeoPoints β
β β component: RecordingInfo:name β component: RecordingInfo:start_time β entity_path: /__properties/info β entity_path: /__properties/info β component: GeoPoints:positions β
β β component_type: Name β component_type: Timestamp β kind: data β kind: data β component_type: LatLon β
β β entity_path: /__properties β entity_path: /__properties β β β entity_path: /__properties/location β
β β kind: data β kind: data β β β kind: data β
ββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ‘
β 4cc4df9667fd4c308c4e3511b5e0da98 β [segment_0] β [1769101329662761000] β [0] β [false] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 473ec142a9464d51a1ee51cac71304ac β [segment_1] β [1769101329954056000] β [1] β [true] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 444dc0d31567473ab6dd5e48c53423e5 β [segment_2] β [1769101329955512000] β [2] β [false] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β c66ec19194ca48648799d1e9d60c6fe6 β [segment_3] β [1769101329956199000] β [3] β [true] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 2ef8804c3979415a8f8d35dc2b2adfa4 β [segment_4] β [1769101329957042000] β [4] β [false] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The segment table contains one row per recording, with each property appearing as a column.
The column metadata exposes the fact that properties are stored under the reserved `/__properties` entity path.
For simplicity, the column names are however prefixed with `property:` instead of the full entity path.
Since the segment table is a [DataFusion](https://datafusion.apache.org/) DataFrame, you can use standard DataFrame operations for further processing and/or data conversion.
For example, this is how the segment table can be filtered based on the values of a custom property:
```python
interesting_segments = segment_table.filter(col("property:info:is_odd")[0])
print(interesting_segments)
```
Output:
```
ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β rerun_segment_id β property:RecordingInfo:name β property:RecordingInfo:start_time β property:info:index β property:info:is_odd β property:location:GeoPoints:positions β
β --- β --- β --- β --- β --- β --- β
β type: Utf8 β type: nullable List[nullable Utf8] β type: nullable List[nullable i64] β type: nullable List[nullable i64] β type: nullable List[nullable bool] β type: nullable List[nullable FixedSizeList[nullable f64; 2]] β
β β archetype: RecordingInfo β archetype: RecordingInfo β component: index β component: is_odd β archetype: GeoPoints β
β β component: RecordingInfo:name β component: RecordingInfo:start_time β entity_path: /__properties/info β entity_path: /__properties/info β component: GeoPoints:positions β
β β component_type: Name β component_type: Timestamp β kind: data β kind: data β component_type: LatLon β
β β entity_path: /__properties β entity_path: /__properties β β β entity_path: /__properties/location β
β β kind: data β kind: data β β β kind: data β
ββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ‘
β 473ec142a9464d51a1ee51cac71304ac β [segment_1] β [1769101329954056000] β [1] β [true] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β c66ec19194ca48648799d1e9d60c6fe6 β [segment_3] β [1769101329956199000] β [3] β [true] β [[46.5, 6.5]] β
ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
## FAQ
### How are properties stored in the recording?
Properties are stored under the reserved `/__properties` entity path and are logged as [static data](https://rerun.io/docs/concepts/logging-and-ingestion/static.md), meaning they have no timeline association.
Logging directly under this entity path is not recommended.
Use the [`rr.send_property()`](https://ref.rerun.io/docs/python/stable/common/property_functions/#rerun.send_property) or [`RecordingStream.send_property()`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.RecordingStream.send_property) API instead.
### How are property columns named?
Property column names follow this general pattern:
```
property:$property_name:$Archetype:$field
```
where `$property_name` is the name provided to `send_property()`, and `$Archetype:$field` is derived from the property data.
For example, a `GeoPoints` archetype logged under the entity `location` appears as `property:location:GeoPoints:positions`.
For built-in properties, the `$property_name` part is omitted, e.g., `property:RecordingInfo:name`.
The `rr.AnyValues` helper logs data without a defined archetype.
As a result, the corresponding columns do not have the `$Archetype` part, e.g., `property:info:index`.
### Are properties visible in dataframe queries?
Yes.
[Dataframe queries](https://rerun.io/docs/concepts/query-and-transform/dataframe-queries.md) can access properties by explicitly including the `/__properties/**` entity path filter, which is excluded by default.
When queried this way, the property column names follow the same rules described above.
```python
df = dataset.filter_contents("__properties/**").reader(index=None)
df = df.sort("property:RecordingInfo:name").select(
"rerun_segment_id", "property:RecordingInfo:name", "property:info:index"
)
print(df)
```
Output:
```
ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ
β rerun_segment_id β property:RecordingInfo:name β property:info:index β
β --- β --- β --- β
β type: Utf8 β type: nullable List[nullable Utf8] β type: nullable List[nullable i64] β
β β archetype: RecordingInfo β component: index β
β β component: RecordingInfo:name β entity_path: /__properties/info β
β β component_type: Name β is_static: true β
β β entity_path: /__properties β kind: data β
β β is_static: true β β
β β kind: data β β
ββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββ‘
β 8e59232215294cb39bc35dad5605bdce β [segment_0] β [0] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 2067b4357dd744648ee0462f39c4de14 β [segment_1] β [1] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 2cf0ff4e3a8c4cf3a0e00e88050cdb01 β [segment_2] β [2] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 780ac12de8694dc7a9a916ccbb8a7218 β [segment_3] β [3] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 99bc7f4dc0c5469c9d154a1cb01e002c β [segment_4] β [4] β
ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ
```
### Can the built-in properties be omitted from recordings?
Yes.
Both [`rr.init()`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.init) and [`RecordingStream()`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.RecordingStream) accept a `send_properties` parameter (default: `True`).
Set it to `False` to prevent the built-in `RecordingInfo` properties from being automatically sent when the recording is created.
# Dataframe queries
Robotic and sensor data is inherently messy:
- Sensors operate at different rates: multiple unaligned data streams
- Data is sparse: not every component has a value at every timestamp
- Multiple timelines coexist: wall clock time, frame numbers, sensor ticks, etc.
Machine learning workloads, on the other hand, rely on aligned rows where each row represents one sample, with a consistent schema and a single index.
Dataframe queries are designed to bridge this gap. They allow you to query arbitrary Rerun data and produce a dataframe as output.
## Where can dataframe queries be used?
Dataframe queries can be used in two contexts:
- **Interactively in the Viewer**: The [dataframe view](https://rerun.io/docs/reference/types/views/dataframe_view.md) displays query results as a table, useful for inspecting raw values and debugging.
- **Programmatically using the Catalog SDK**: The [`DatasetEntry`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry) object provides API to filter and query datasets and turn them into dataframes.
## Understanding dataframe queries
Let's use an example to illustrate how dataframe queries work.
Dataframe queries run against datasets stored on a [catalog server](https://rerun.io/docs/concepts/how-does-rerun-work.md).
We can create a demo recording and load it into a temporary local catalog using the following code:
```python
# create some data
times = list(range(64))
scalars = [math.sin(t / 10.0) for t in times]
# log the data to a temporary recording
with rr.RecordingStream("rerun_example_dataframe_query") as rec:
rec.save(RRD_PATH)
rec.send_columns(
"/data",
indexes=[rr.TimeColumn("step", sequence=times)],
columns=rr.Scalars.columns(scalars=scalars),
)
```
We can then perform a dataframe query (against the local open-source catalog server included in Rerun):
```python
# load the demo recording in a temporary catalog
with rr.server.Server(datasets={"dataset": [RRD_PATH]}) as server:
# obtain a dataset from the catalog
dataset = server.client().get_dataset("dataset")
# (optional) filter interesting data
dataset_view = dataset.filter_contents("/data")
# obtain a DataFusion dataframe
df = dataset_view.reader(index="step")
# (optional) filter rows using DataFusion expressions
df = df.filter(col("/data:Scalars:scalars")[0] > 0.95)
# execute the query
print(df) # or convert to Pandas, Polars, PyArrow, etc.
```
This should produce an output similar to:
```
ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ
β rerun_segment_id β step β /data:Scalars:scalars β
β --- β --- β --- β
β type: Utf8 β type: nullable i64 β type: nullable List[nullable f64] β
β β index_name: step β archetype: Scalars β
β β kind: index β component: Scalars:scalars β
β β β component_type: Scalar β
β β β entity_path: /data β
β β β kind: data β
ββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββ‘
β 5712205b356b470e8d1574157e55f65e β 13 β [0.963558185417193] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 5712205b356b470e8d1574157e55f65e β 14 β [0.9854497299884601] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 5712205b356b470e8d1574157e55f65e β 15 β [0.9974949866040544] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 5712205b356b470e8d1574157e55f65e β 16 β [0.9995736030415051] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 5712205b356b470e8d1574157e55f65e β 17 β [0.9916648104524686] β
ββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ€
β 5712205b356b470e8d1574157e55f65e β 18 β [0.9738476308781951] β
ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ
```
Let's unpack what happened here:
- **Catalog required**: We use `rr.server.Server()` to spin up a temporary local catalog. In production, you might connect to a Rerun Hub deployment instead. We then obtain the dataset to be queried from the catalog.
- **Content filtering**: The `filter_contents()` method restricts the scope of the query to specific entities. This affects which columns are returned, but may also change which rows are returned since rows are only produced where at least one filtered column has data (see [How are rows produced?](#how-are-rows-produced-by-dataframe-queries)).
- **Reader produces a lazy dataframe**: The `reader(index=β¦)` method returns a [DataFusion](https://datafusion.apache.org/) dataframe. The `index` parameter specifies which timeline drives row generation: a row is produced for each unique value of this index where data exists. The returned dataframe doesn't execute until it is collected.
- **Filtering/aggregation/joining/etc.**: The standard suite of dataframe operations is provided by DataFusion. Here we use `filter()` to filter rows based on the data. Again, these are lazy operations that only build a query plan.
- **Execution**: The `print(df)` implicitly executes the dataframe's query plan and returns the final result. The same would happen when converting to dataframe for other frameworks (Pandas, Polars, PyArrow, etc.).
## FAQ
### How are rows produced by dataframe queries?
A row is produced for each distinct index (or timeline) value for which there is at least one value in the filtered content.
For example, if you filter for entities `/camera` and `/lidar`, and `/camera` has data at timestamps [1, 2, 3] while `/lidar` has data at [2, 4], the output will have rows for timestamps [1, 2, 3, 4]. Columns without data at a given timestamp will contain null values (unless sparse fill is enabled).
### What is the difference between dataset's `filter_contents()` and DataFusion's `select()`?
At first glance, both methods control which columns appear in the result. However, they differ in an important way:
- **`filter_contents()`** restricts which entities are considered for row generation. This affects both which columns *and* which rows are returned.
- **`select()`** is a DataFusion operation that only filters columns *after* rows have been determined. It does not affect row generation.
Building on the previous example, if `/camera` has data at timestamps [1, 2, 3] and `/lidar` has data at [2, 4]:
```python
# Rows at [1, 2, 3] with only /camera columns
dataset.filter_contents("/camera").reader(index="timestamp")
# Rows at [1, 2, 3, 4] with only /camera columns
# (null values at timestamp 4 where /camera has no data)
dataset.filter_contents(["/camera", "/lidar"]).reader(index="timestamp").select("/camera")
```
### How are segments handled by dataframe queries?
When querying a dataset with multiple [segments](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md), the query is applied on a segment-by-segment basis. This means:
- Latest-at semantics do not cross segment boundaries. Each segment is queried independently.
- The output includes a `rerun_segment_id` column identifying which segment each row comes from.
- Use `filter_segments()` on a dataset or dataset view to restrict the query to specific segment IDs.
### How is static data queried?
As a reminder, [static data](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) has no associated timeline and represents values that don't change over time. When data is logged as static to a column, it is considered valid for all timelines and for all times, overriding any temporal data otherwise logged to the same column.
The consequence of this is that static data cannot, by itself, generate rows. However, for rows that are generated by other (temporal) data, static data will show up in their respective columns provided they are part of the filtered content.
In practice, this can cause performance and/or memory issues when the same large static data is yielded in every row.
For this reason, it may be preferable to filter static columns out (e.g. using `filter_contents()`) and query the static data separately.
Querying static data only can also be useful for retrieving configuration, calibration data, or other time-invariant information.
This is achieved by setting the index to `None`:
```python
df = dataset.reader(index=None)
```
The returned dataframe contains a single row with all the static data from the filtered content.
### How do dataframe queries achieve resampling?
By default, rows are produced only at index values where data exists. To sample at specific timestamps (even if no data exists there), use the `using_index_values` parameter combined with `fill_latest_at=True`:
```python
# Sample at fixed 10Hz (100ms intervals)
timestamps = np.arange(start_time, end_time, np.timedelta64(100, "ms"))
df = dataset.reader(
index="timestamp",
using_index_values=timestamps,
fill_latest_at=True,
)
```
- `using_index_values` specifies the exact timestamps to sample
- `fill_latest_at=True` fills null values with the most recent data (latest-at/forward fill semantics)
For a complete example, see the [Time-align data](https://rerun.io/docs/howto/query-and-transform/time_alignment.md) how-to.
## Additional resources
- [π Python Catalog SDK reference](https://ref.rerun.io/docs/python/stable/common/catalog/)
- [`DatasetEntry`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry)
- [`DatasetView`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetView)
- [`reader()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetView.reader)
- [`filter_contents()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry.filter_contents)
- [`filter_segments()`](https://ref.rerun.io/docs/python/stable/common/catalog/#rerun.catalog.DatasetEntry.filter_segments)
- [Dataframe view](https://rerun.io/docs/reference/types/views/dataframe_view.md) for visualizing query results in the Viewer
- [Query semantics & partial updates](https://rerun.io/docs/concepts/logging-and-ingestion/latest-at.md) for understanding latest-at and range queries
# Static data
The Rerun SDK allows you to store data as _static_. Static data belongs to all timelines (existing ones, and ones not yet created) and shadows any temporal data of the same type on the same entity.
That is, any time you log static data to an entity path, all past, present and future temporal data on that same entity path and component is _semantically_ discarded in favor of the static one (which doesn't necessarily mean that it is _physically_ discarded, more on that below).
## How to store static data?
Internally, all data in Rerun is stored as chunks of columns. Specifically, each chunk holds zero or more time columns (the indices), and zero or more component columns (the data).
Static data is data that lives in a chunk whose set of time columns is the empty set.
The easiest way to create such chunks is by using the `log` family of methods, which exposes a `static` flag where appropriate:
```python
rr.log("skybox", generate_skybox_mesh(), static=True)
```
The same can be achieved using the `send_columns` API by simply leaving the time column set empty:
```python
rr.send_columns("skybox", indexes=[], columns=generate_skybox_mesh())
```
(Using `send_columns` that way is rarely useful in practice, but is just a logical continuation of the data model.)
## When should I use static data?
There are two broad categories of situations where you'd want to use static data: scene setting and memory savings.
### Scene setting
Often, you'll want to store data that isn't part of normal data capture, but sets the scene for how it should be shown.
For instance, if you are logging cars on a street, perhaps you want to always show a street mesh as part of the scenery, and for that it makes sense for that data to be static.
```python
rr.log("skybox", generate_skybox_mesh(), static=True)
```
The alternative would be to log that data at the beginning of every relevant timeline, which can be very problematic as the set of timelines might not even be known before runtime.
Similarly, [coordinate systems](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md) or [annotation context](https://rerun.io/docs/concepts/visualization/annotation-context.md) are typically stored as static.
### Memory savings
When you store _temporal_ data in Rerun, it is always appended to the existing dataset: there is no such thing as overwriting temporal data. The dataset only grows, it never shrinks.
To compensate for that, the Rerun viewer has a [garbage collection mechanism](https://rerun.io/docs/howto/visualization/limit-ram.md) that will drop the oldest data from the store when memory becomes scarce.
For example, the following snippet stores 10 images at index `4` on the `frame` [timeline](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md):
```python
rr.set_time("frame", sequence=4)
for _ in range(10):
rr.log("camera/image", camera.save_current_frame())
```
All these images are actually stored, and all of them can be visualized in the viewer independently, even though they share the same index.
Contrary to temporal data, static data is **never** garbage collected⦠but it can actually be overwritten!
_Semantically_, only a single piece of static data can exist at a given time for a specific component on a specific entity.
In the following snippet, only the data from latest log call (in execution order) will be inspectable in the viewer:
```python
for _ in range(10):
rr.log("camera/image", camera.save_current_frame(), static=True)
```
In practice, the Rerun datastore will rely on these semantics to physically drop the superfluous static data where possible, therefore drastically reducing memory costs. See ["Understanding storage costs"](#understanding-storage-costs) for more information.
## Understanding storage costs
In ["Memory savings"](#memory-savings), we mentioned that the following snippet _semantically_ stores a single image:
```python
for _ in range(10):
rr.log("camera/image", camera.save_current_frame(), static=True)
```
How these semantics actually translate to physical storage depends on the context.
### In recordings
Rerun recordings (`.rrd` files) are just streams of binary messages: they have no semantics whatsoever, therefore they don't know what static means and can't do anything about it.
If you were to log the snippet above to a file (using e.g. `rr.save()`), you'd find that the recording does in fact contains your 10 images.
If you wanted the recording file itself to only contain a single static value, you would need to either:
* Stream the data to the viewer, and then save the recording directly out of the viewer using `Menu > Save recording` (or the equivalent palette command).
* Manually recompact your recording using the [Rerun CLI](https://rerun.io/docs/reference/cli.md) so that the data overwrite semantics can get appropriately applied, e.g.: `rerun rrd optimize -o compacted.rrd myrecording.rrd`.
### In the viewer
The data store that backs the Rerun viewer natively understands these temporal/garbage-collected vs. static/overwritten semantics.
If you were to log the snippet above directly to the Rerun viewer (using e.g. `rr.connect_grpc()`), you'd notice that the viewer's memory usage stays constant: the data is automatically being overwritten as new updates come in.
For data where you don't need to keep track of historical values, this effectively to logs its new values indefinitely.
In the following example, you can see our [face tracking example]() indefinitely tracking my face while maintaining constant memory usage by logging all data as static:
# Entities and Components
## Data model
The core of Rerun's data model is inspired by the ideas of the [Entity Component System (ECS)](https://en.wikipedia.org/wiki/Entity_component_system) architecture pattern. In
short, an ECS is a composition-oriented framework in which *entities* represent generic objects while *components* describe
data associated with those entities.
* *Entities* are the "things" that you log with the [`rr.log()`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log)function. They are represented by the
[*entity path*](https://rerun.io/docs/concepts/logging-and-ingestion/entity-path.md) string which is passed as first argument.
* *Components*, however, are what contains the data that is associated with those "things". For example, position, color,
pixel data, etc.
Entities are like folders, and components are like files.
Additionally, the Rerun SDKs expose two additional concepts:
* *Archetypes* are coherent set of components corresponding to primitive such as 2D points or 3D boxes. In the Rerun SDKs, archetypes take the form of builder objects that assist with the creation of such component sets. They are meant as high-level, convenience helpers that can be bypassed entirely if/when required by advanced use-cases.
* *Datatypes* are regular data structures that components occasionally rely on when fundamental data types (`float`, `uint32`, etc.) are not sufficient.
### Logging and viewing data
All the data that you log within Rerun is mapped to the concepts of entities and components.
For example, consider the case of logging a point:
```python
rr.log("my_point", rr.Points2D([32.7, 45.9], colors=[255, 0, 0]))
```
This statement uses the [`rr.Points2D`](https://ref.rerun.io/docs/python/stable/common/archetypes/#rerun.archetypes.Points2D) archetype.
Internally, this archetype builds a set of, in this case, two components:
* `Points2D:positions` of type [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
* `Points2D:colors` of type [`Color`](https://rerun.io/docs/reference/types/components/color.md).
Then, the `rr.log()` function records these two components and associate them with the `"my_point"` entity.
Later, the View for spatial types queries the data store for all the entities that have a `Points2D:positions` component.
In this case it would find the "my_point" entity. This query additionally returns the `Points2D:colors` component because that
component is associated with the same entity. These two components are recognized as corresponding to the `Points2D` archetype (via metadata attached to the components), which informs the Viewer on how to display the corresponding entity.
See the [Types](https://rerun.io/docs/reference/types.md) reference for a list of [archetypes](https://rerun.io/docs/reference/types/archetypes.md),
[components](https://rerun.io/docs/reference/types/components.md), and [datatypes](https://rerun.io/docs/reference/types/datatypes.md).
### Adding custom data
Although both the SDKs' archetype objects and the view are based on the same archetype definition (and are actually implemented using code that is automatically generated based on that definition), they both operate on arbitrary collection
of components. Neither the SDKs nor the Viewer enforce or require that an entity should contain a *specific* set of component.
The Rerun Viewer will display any data in a generic form, but its views will only work on sets of components it can
make sense of.
Your entity could have any number of additional components as well. This isn't a problem. Any components that
aren't relevant to the scene that the view is drawing are safely ignored. Also, Rerun even allows you to log your
own set of components, bypassing archetypes altogether.
In Python, the [rr.AnyValues](https://ref.rerun.io/docs/python/stable/common/custom_data/#rerun.AnyValues) helper object can be used to add custom component(s) to an archetype:
```python
"""Log extra values with a `Points2D`."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_extra_values", spawn=True)
rr.log(
"extra_values",
rr.Points2D([[-1, -1], [-1, 1], [1, -1], [1, 1]]),
rr.AnyValues(
confidence=[0.3, 0.4, 0.5, 0.6],
),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(
x_range=[-1.5, 1.5], y_range=[-1.5, 1.5]
)
)
)
```
It can also be used log an entirely custom set of components:
```python
"""Log arbitrary data."""
import rerun as rr
rr.init("rerun_example_any_values", spawn=True)
rr.log(
"any_values",
rr
.AnyValues(
# Using arbitrary Arrow data.
homepage="https://www.rerun.io",
repository="https://github.com/rerun-io/rerun",
)
# Using Rerun's builtin components.
.with_component_override(
"confidence", rr.components.ScalarBatch._COMPONENT_TYPE, [1.2, 3.4, 5.6]
)
.with_component_override(
"description", rr.components.TextBatch._COMPONENT_TYPE, "Bla bla blaβ¦"
),
)
```
For more complex use-cases, custom objects implementing the `rr.AsComponents` protocol can be used. For Rust, the `rerun::AsComponents` trait must be implemented:
```python
"""Shows how to implement custom archetypes and components."""
from __future__ import annotations
import argparse
from typing import Any
import numpy as np
import numpy.typing as npt
import pyarrow as pa
import rerun as rr
class ConfidenceBatch(rr.ComponentBatchMixin): # type: ignore[misc]
"""A batch of confidence data."""
def __init__(self: Any, confidence: npt.ArrayLike) -> None:
self.confidence = confidence
def as_arrow_array(self) -> pa.Array:
"""The arrow batch representing the custom component."""
return pa.array(self.confidence, type=pa.float32())
class CustomPoints3D(rr.AsComponents): # type: ignore[misc]
"""A custom archetype extending the builtin `Points3D` with extra data."""
def __init__(
self: Any, positions: npt.ArrayLike, confidences: npt.ArrayLike
) -> None:
self.points3d = rr.Points3D(positions)
self.confidences = ConfidenceBatch(confidences).described(
rr.ComponentDescriptor(
"user.CustomPoints3D:confidences",
archetype="user.CustomPoints3D",
component_type="user.Confidence",
)
)
def as_component_batches(self) -> list[rr.DescribedComponentBatch]:
return [
# The components from Points3D
*self.points3d.as_component_batches(),
# Custom confidence data
self.confidences,
]
def log_custom_data() -> None:
lin = np.linspace(-5, 5, 3)
z, y, x = np.meshgrid(lin, lin, lin, indexing="ij")
point_grid = np.vstack([x.flatten(), y.flatten(), z.flatten()]).T
rr.log(
"left/my_confident_point_cloud",
CustomPoints3D(
positions=point_grid,
confidences=[42],
),
)
rr.log(
"right/my_polarized_point_cloud",
CustomPoints3D(
positions=point_grid, confidences=np.arange(0, len(point_grid))
),
)
def main() -> None:
parser = argparse.ArgumentParser(
description="Logs rich data using the Rerun SDK."
)
rr.script_add_args(parser)
args = parser.parse_args()
rr.script_setup(args, "rerun_example_custom_data")
log_custom_data()
rr.script_teardown(args)
if __name__ == "__main__":
main()
```
### Empty entities
An entity without components is nothing more than an identity (represented by its entity
path). It contains no data, and has no type. When you log a piece of data, all that you are doing is setting the values
of one or more components associated with that entity.
## ECS systems
There is a third concept we haven't touched on: *systems* are processes which operate on the entities based on the components they possess.
Rerun is still settling on the exact form of formalized systems and outside of Rust Viewer code it is not yet possible to write your own systems. However, views work under the hood using a variety of systems. For more information see the [Extend the Viewer in Rust](https://rerun.io/docs/howto/visualization/extend-ui.md) section.
# Events and Timelines
## Timelines
Each piece of logged data is associated with one or more timelines.
The logging SDK can automatically create two timelines for you:
* `log_time` - a temporal timeline with the time of the log call. Enabled by default; opt-out via the `RERUN_LOG_TIME` environment variable or `set_log_time_enabled`.
* `log_tick` - a sequence timeline with the sequence number of the log call. Disabled by default; opt-in via the `RERUN_LOG_TICK` environment variable or `set_log_tick_enabled`.
You can use the `set_time` function (Python reference: [set_time](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.set_time)) to associate logs with other timestamps on other timelines. For example:
```python
for frame in read_sensor_frames():
rr.set_time("frame_idx", sequence=frame.idx)
rr.set_time("sensor_time", timestamp=frame.timestamp)
rr.log("sensor/points", rr.Points3D(frame.points))
```
This will add the logged points to the timelines `frame_idx` and `sensor_time`, as well as the automatic `log_time` timeline (and `log_tick`, if you opted in).
You can then choose which timeline you want to organize your data along in the expanded timeline view in the bottom of the Rerun Viewer.
### How to log precise times
Rerun supports three types of indices, all encoded as `i64`:
* Sequential
* Timestamp (nanoseconds since Unix epoch)
* Timedelta/duration (nanoseconds)
Here's how you use them:
```python
"""Set different types of indices."""
from datetime import datetime
import numpy as np
import rerun as rr
rr.init("rerun_example_different_indices", spawn=True)
rr.set_time("frame_nr", sequence=42)
rr.set_time("elapsed", duration=12) # elapsed seconds
rr.set_time("time", timestamp=1_741_017_564) # Seconds since unix epoch
rr.set_time("time", timestamp=datetime.fromisoformat("2025-03-03T15:59:24"))
rr.set_time(
"precise_time", timestamp=np.datetime64(1_741_017_564_987_654_000, "ns")
) # Nanoseconds since unix epoch
# All following logged data will be timestamped with the above times:
rr.log("points", rr.Points2D([[0, 0], [1, 1]]))
```
### Reset active timeline & differing data per timeline
You can clear the active timeline(s) at any point using `reset_time`.
This can be particularly useful when you want to log different data for individual timelines as illustrated here:
```python
"""Log different data on different timelines."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_different_data_per_timeline", spawn=True)
rr.set_time("blue timeline", sequence=0)
rr.set_time("red timeline", duration=0.0)
rr.log("points", rr.Points2D([[0, 0], [1, 1]], radii=rr.Radius.ui_points(10.0)))
# Log a red color on one timeline.
rr.reset_time() # Clears all set timeline info.
rr.set_time("red timeline", duration=1.0)
rr.log("points", rr.Points2D.from_fields(colors=[255, 0, 0]))
# And a blue color on the other.
rr.reset_time() # Clears all set timeline info.
rr.set_time("blue timeline", sequence=1)
rr.log("points", rr.Points2D.from_fields(colors=[0, 0, 255]))
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 2], y_range=[-1, 2])
)
)
```
On one timeline the points will appear blue, on the other they appear red.
### Sending many time points at once
To get full control over the logged timelines you can use [`send_columns`](https://rerun.io/docs/howto/logging-and-ingestion/send-columns.md).
This is often a lot more efficient when you already have a chunk of temporal data, e.g. some sensor value over time.
## Events
An _event_ refer to an instance of logging one or more component batches to one or more timelines. In the viewer, the Time panel provide a graphical representation of these events across time and entities.
## Static data
The [`rr.log()`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log) function has a `static=False` default argument.
If `static=True` is used instead, the data logged becomes *static*. Static data belongs to all timelines (existing ones, and ones not yet created) and shadows any temporal data of the same type on the same entity.
This is useful for data that isn't part of normal data capture, but sets the scene for how it should be shown.
For instance, if you are logging cars on a street, perhaps you want to always show a street mesh as part of the scenery, and for that it makes sense for that data to be static.
Similarly, [coordinate systems](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md) or [annotation context](https://rerun.io/docs/concepts/visualization/annotation-context.md) are typically static.
You can read more about static data in the [dedicated section](https://rerun.io/docs/concepts/logging-and-ingestion/static.md).
# The Entity Path Hierarchy
## Entity paths
As mentioned in the [Entity Component](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) overview, all entities within Rerun have a unique _entity path_.
The first argument to the `log()` function is this path. Each time you log to a specific entity path you will update the entity, i.e. log a new instance of it along the timeline.
It is possible to log multiple types of archetypes on the same entity path, but you should generally avoid mixing different kinds of geometric primitive. For example, logging a [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md) point cloud on an entity path where a [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md) was previously logged would overwrite the mesh's [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md) component with the point cloud's, but would leave the `triangle_indices` component untouched. The Rerun Viewer would likely be unable to display the result. See the [Entity Component](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) section for more information.
There _are_ valid reasons to logs different kinds of archetypes to the same entity path, though. For example, it's common to log a [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md) along with some geometry it relates to (see the [Transforms & Coordinate Frames](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md) for more info).
Rerun treats entity paths as being arranged in a hierarchy with the `/` character acting as a separator between path
elements. The conventional path semantics including concepts of *root* and *parent*/*child* generally apply.
When writing paths in logging APIs the leading `/` is usually omitted.
In the file path analogy, each entity is a folder, and a component is a file.
This implies that any entity in a hierarchy can contain components.
For example (this uses the Python SDK but the same applies for all supported languages):
```python
rr.log("image", rr.Image(img))
rr.log("image/points", rr.Points2D(points))
```
It is also acceptable to leave implicitly "empty" entities in your paths as well.
```python
rr.log("camera/image", rr.Image(img))
rr.log("camera/image/detections/points", rr.Points2D(points))
```
Nothing needs to be explicitly logged to `"camera"` or `"camera/image/detection"` to make the above valid.
In other words, the `log` call is akin to creating a folder with `mkdir -p` and then writing files (components) to it.
Existing components of the same name will be overwritten.
### Path parts
Each "part" of a path must be a non-empty string. Any character is allowed, but special characters need to be escaped using `\`.
Characters that need NOT be escaped are letters, numbers, and underscore, dash, and dot (`_`, `-`, `.`).
Any other character should be escaped, including symbols (`\:`, `\$`, β¦) and whitespace (`\ `, `\n`, `\t`, β¦).
You can insert an arbitrary unicode code point into an entity path using `\u{262E}`.
So for instance, `world/3D/My\ Image.jpg/detection` is a valid path (note the escaped space!).
> [!WARNING]
> Even though entity paths are somewhat analogous to file paths, they are NOT the same. `..` does not mean "parent folder", and you are NOT intended to pass a file path as an entity path (especially not on Windows, which use `\` as a path separator).
### Path hierarchy functions
Path hierarchy plays an important role in a number of different functions within Rerun:
* With the [Transform System](https://rerun.io/docs/concepts/logging-and-ingestion/transforms.md) the `transform` component logged to any entity always describes
the relationship between that entity and its direct parent.
* When resolving the meaning of [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md) and [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md) components, Rerun uses the [Annotation Context](https://rerun.io/docs/concepts/visualization/annotation-context.md) from the nearest ancestor in the hierarchy.
* When adding data to [Blueprints](https://rerun.io/docs/reference/viewer/blueprints.md), it is common to add a path and all of its descendants.
* When using `rr.log("entity/path", rr.Clear(recursive=True))`, it marks an entity *and all of its descendants* as being cleared.
* In the future, it will also be possible to use path-hierarchy to set default-values for descendants
([#1158](https://github.com/rerun-io/rerun/issues/1158)).
### Reserved paths
The path prefix `__` is considered reserved for use by the Rerun SDK itself and should not be used for logging
user data. This is where Rerun will log additional information such as properties (`__properties`) and warnings
(`__warnings`).
# Component Batches
In the Rerun data model, the value of a given component at a given point in time is always itself a list β or a _batch_ β of values.
Consider this example:
```python
rr.log("/data", rr.Points3D(positions=[0.0, 0.0, 0.0]))
```
For convenience, the [`rr.Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md) archetype accepts a single position, but what actually happens is that the corresponding [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md) component is logged as a batch of length 1.
So the following log calls are equivalent:
```python
single_point = [0.0, 0.0, 0.0]
rr.log("/data", rr.Points3D(positions=single_point)
rr.log("/data", rr.Points3D(positions=[single_point])
```
Logging larger batches is obviously possible:
```python
rr.log("/data", rr.Points3D(positions=[[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]]))
```
The ability to log data as batches is useful in many cases, such as point clouds (as in the above example), bounding boxes for detected objects, tracked keypoints in a skeleton, or individual joint values for a robot arm.
This is also why, in the logging APIs, the majority of archetypes are named with the plural form, like `rr.Points3D` above.
An individual value within a batch is called an _instance_.
## Component batches are immutable
When data is logged to a component for a given time point, the corresponding batch is immutable.
This means that additional instances cannot be appended to it, and existing instances cannot be modified.
The entire batch must be logged again, and this will replace the previous one.
Note that when data is logged multiple times for the same component and at the same time point, the last logged batch will be used, but the previously logged batches will remain in storage.
## Instance joining semantics
Components are typically logged as part of archetypes, which are semantic groupings of related components (see [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md)).
Often, archetypes have instance joining semantics.
This means that the nth instance of one of the components relates to the nth instance of other components.
For example, this is the case of [`rr.Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md): the nth value of its `colors` field applies to the nth value of its `positions` field.
### Instance clamping
Such archetypes typically have a required component that acts as the _primary component_.
That's the component which defines how many logical instances the logged archetype represents.
For `rr.Points3D`, the primary component is [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md).
Its batch size determines how many points will be visible in the viewer.
For components other than the primary component:
- if they have more instances, the additional instances are ignored by the viewer;
- if they have fewer instances, the last instance is repeated as required.
We refer to the latter case as _clamping semantics_, which can also be seen as a left-join using the primary component.
This enables natural logging calls such as the following:
```python
rr.log("/data", rr.Points3D(positions=[[0.0, 0.0, 0.0], [1.0, 1.0, 1.0], [2.0, 2.0, 2.0]], radii=0.5))
```
Here, an N=3 batch of positions is logged, along with a batch of N=1 radii.
That unique radius value is clamped to the three positions and thus applies to all three points when displayed in the viewer.
### Instance joining and latest-at semantics
Instance joining applies to the _current_ value of the components being displayed in the viewer.
It is worth remembering that the [latest-at semantics](https://rerun.io/docs/concepts/logging-and-ingestion/latest-at.md) still apply, which means that joined components do not need to be logged at the same time.
For example, one might log a point cloud with positions and colors at the beginning of a recording, and later only log updated positions.
The viewer will always look up for the "last" colors that were logged ("latest at" semantics) and use them for display.
### Instance joining is not universal
Note that instance joining semantics are not universal.
Some archetypes don't use it, or use it partially.
For example, the [`rr.Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md) archetype has a `vertex_positions` required component, which defines the number of vertices in the mesh.
Some components have instance joining semantics with `vertex_positions`, including `vertex_colors` and `vertex_texcoords`.
However, some other components do not, including `triangle_indices` which contains triplets of indices into the `vertex_positions` batch and defines the triangles to be displayed.
## Storage
Internally, component data is stored as [Arrow List arrays](https://arrow.apache.org/docs/format/Columnar.html#variable-size-list-layout) within [chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md).
Each row of the list array corresponds to a single time point, and the values in each row correspond to the component batch.
The Rerun data model exploits the fact that list arrays can have different lengths in each row to allow component batches to have different lengths at each time point.
This design choice is most visible when [querying Rerun data](https://rerun.io/docs/concepts/query-and-transform/dataframe-queries.md).
The returned dataframes will always have the `ListArray` datatype for component columns, even if the underlying columns contain a single value per row, or all rows (or batches) have the same length.
## See also
[`send_columns`](https://rerun.io/docs/howto/logging-and-ingestion/send-columns.md) lets you efficiently send many batches of data in one log call.
# Transforms & Coordinate Frames
Rerun comes with built-in support for modeling spatial relationships between entities.
This page details how the [different archetypes](https://rerun.io/docs/reference/types/archetypes#transforms) involved interact with each other and explains how transforms are set up in Rerun.
## Transforms
### Entity path transforms
The [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d) archetype allows you to specify how one coordinate system relates to another through translation, rotation, and scaling.
The simplest way to use transforms is through [entity path hierarchies](https://rerun.io/docs/concepts/logging-and-ingestion/entity-path.md), where each transform describes the relationship between an entity and its parent path.
Note that by default, all entities are connected via identity transforms.
```python
"""Logs a simple transform hierarchy."""
import rerun as rr
rr.init("rerun_example_transform3d_hierarchy_simple", spawn=True)
# Log entities at their hierarchy positions.
rr.log(
"sun",
rr.Ellipsoids3D(
half_sizes=[1, 1, 1], colors=[255, 200, 10], fill_mode="solid"
),
)
rr.log(
"sun/planet",
rr.Ellipsoids3D(
half_sizes=[0.4, 0.4, 0.4], colors=[40, 80, 200], fill_mode="solid"
),
)
rr.log(
"sun/planet/moon",
rr.Ellipsoids3D(
half_sizes=[0.15, 0.15, 0.15], colors=[180, 180, 180], fill_mode="solid"
),
)
# Define transforms - each describes the relationship to its parent.
rr.log(
"sun/planet", rr.Transform3D(translation=[6.0, 0.0, 0.0])
) # Planet 6 units from sun.
rr.log(
"sun/planet/moon", rr.Transform3D(translation=[3.0, 0.0, 0.0])
) # Moon 3 units from planet.
```
In this hierarchy:
- The `sun` entity exists at the origin of its own coordinate system
- The `sun/planet` transform places the planet 6 units from the sun, along the x-axis
- The `sun/planet/moon` transform places the moon 3 units along x away from the planet
This creates a transform hierarchy where transforms propagate down the entity tree. The moon's final position in the sun's coordinate system is 9 units away (6 + 3),
because the transforms are applied sequentially.
### Named transform frames
While entity path hierarchies work well for many cases, sometimes you need more flexibility in organizing your transforms.
In particular, for anyone familiar with ROS, we recommend using named transform frames as it allows you to model
your data much closer to how it would be defined when using ROS' [tf2](https://wiki.ros.org/tf2) library.
By explicitly specifying transform frames, you can decouple spatial relationships from the entity hierarchy.
Instead of relying on entity path relationships, each entity is first associated with a named transform frame using
the [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame) archetype.
The geometric relationship between two transform frames is then determined by logging [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d)
with `child_frame` and `parent_frame` parameters set to their respective names.
```python
"""Logs a simple transform hierarchy with named frames."""
import rerun as rr
rr.init("rerun_example_transform3d_hierarchy_named_frames", spawn=True)
# Define entities with explicit coordinate frames.
rr.log(
"sun",
rr.Ellipsoids3D(
half_sizes=[1, 1, 1], colors=[255, 200, 10], fill_mode="solid"
),
rr.CoordinateFrame("sun_frame"),
)
rr.log(
"planet",
rr.Ellipsoids3D(
half_sizes=[0.4, 0.4, 0.4], colors=[40, 80, 200], fill_mode="solid"
),
rr.CoordinateFrame("planet_frame"),
)
rr.log(
"moon",
rr.Ellipsoids3D(
half_sizes=[0.15, 0.15, 0.15], colors=[180, 180, 180], fill_mode="solid"
),
rr.CoordinateFrame("moon_frame"),
)
# Define explicit frame relationships.
rr.log(
"planet_transform",
rr.Transform3D(
translation=[6.0, 0.0, 0.0],
child_frame="planet_frame",
parent_frame="sun_frame",
),
)
rr.log(
"moon_transform",
rr.Transform3D(
translation=[3.0, 0.0, 0.0],
child_frame="moon_frame",
parent_frame="planet_frame",
),
)
# Connect the viewer to the sun's coordinate frame.
# This is only needed in the absence of blueprints since a default view will
# typically be created at `/`.
rr.log("/", rr.CoordinateFrame("sun_frame"), static=True)
```
Note that unlike in ROS, you can log your transform relationship on _any_ entity.
> [!NOTE]
> A current limitation to this is that once a `Transform3D` (or `Pinhole`) relating two frames has been logged to an entity, this particular relation may no longer be logged on any other entity.
> An exception to this rule is [static data](https://rerun.io/docs/concepts/logging-and-ingestion/static.md): if you log a frame to frame relationship on an entity with static time, you can later on use a different entity for temporal information.
> This is useful to specify "default" transforms without yet knowing what timeline and paths are going to be used for temporal transforms.
Named transform frames have several advantages over entity path based hierarchies:
* topology may change over time
* association of entities with coordinate frames is explicit and may changed over time (it can also be [overridden via blueprint](https://rerun.io/docs/concepts/visualization/customize-views.md))
* several entities may be associated with the same frame
* frees up entity paths for semantic rather than geometric organization
### Entity hierarchy based transforms under the hood
Under the hood, Rerun's entity path hierarchies actually use the same transform frame system as named frames.
For each entity path, an associated transform frame with the prefix `tf#` is automatically created:
for example, an entity `/world/robot` gets frame `tf#/world/robot`.
Path based hierarchies are then established by defaults the Viewer uses (also referred to as fallbacks):
Given an entity `/world/robot`:
* if no `CoordinateFrame::frame` is specified, it automatically defaults to `tf#/world/robot`
* if no `Transform3D::child_frame` is specified, it automatically defaults to `tf#/world/robot`
* if no `Transform3D::parent_frame` is specified, it automatically defaults to the parent's implicit frame, `tf#/world`
The only special properties these implicit frames have over their named counterparts is that they
have implicit identity relationships.
#### Example
Given these entities:
```python
rr.log("robot", rr.Transform3D(translation=[1, 0, 0]))
rr.log("robot/arm", rr.Transform3D(translation=[0, 1, 0]))
rr.log("robot/arm/gripper", rr.Points3D([0, 0, 0]))
```
Rerun will interpret this _as-if_ it was logged with the named transform frames like so:
```python
rr.log(
"robot",
rr.CoordinateFrame("tf#/robot"),
rr.Transform3D(translation=[1, 0, 0], child_frame="tf#/robot", parent_frame="tf#/"),
)
rr.log(
"robot/arm",
rr.CoordinateFrame("tf#/robot/arm"),
rr.Transform3D(translation=[0, 1, 0], child_frame="tf#/robot/arm", parent_frame="tf#/robot"),
)
rr.log("robot/arm/gripper", rr.CoordinateFrame("tf#/robot/arm/gripper"), rr.Points3D([0, 0, 0]))
```
#### Mixing named and implicit transform frames
We generally do not recommend mixing named and implicit transform frames since it can get confusing,
but doing so works seamlessly and can be useful if necessary.
Example:
```python
rr.log("robot", rr.Transform3D(translation=[1, 0, 0]))
rr.log(
"arm",
rr.Transform3D(translation=[0, 1, 0], parent_frame="tf#/robot", child_frame="arm_frame"),
rr.CoordinateFrame("arm_frame"),
)
rr.log("gripper", rr.Points3D([0, 0, 0]), rr.CoordinateFrame("arm_frame"))
```
## Other transform types
### Pinhole projections
In Rerun, pinhole cameras are not merely another archetype that can be visualized,
they are also treated as spatial relationships that define projections from 3D spaces to 2D subspaces.
This unified approach allows the Viewer to handle both traditional 3D-to-3D transforms and 3D-to-2D projections.
The [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole) archetype defines this projection relationship through its intrinsic matrix (`image_from_camera`) and resolution.
Both implicit & named coordinate frames are supported, exactly as on [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
With the right setup, pinholes allow a bunch of powerful visualizations:
* the pinhole glyph itself in 3D views
* 2D in 3D: all 2D content that is part of the pinhole's transform subtree
* 3D in 2D: if the pinhole is at the origin of the view, 3D objects can be projected through pinhole camera into the view.
* Both the [nuscenes](https://rerun.io/examples/robotics/nuscenes_dataset) and [arkit](https://rerun.io/examples/spatial-computing/arkit_scenes) examples make use of this
If a transform frame relationship has both a pinhole projection & regular transforms (in this context often regarded as the camera extrinsics),
the regular transform is applied first.
#### Example: 3D scene with 2D projections
Here's how to set up a 3D scene with pinhole cameras that create 2D projections:
In this example, the 3D objects (box and points) are automatically projected into the 2D camera view,
demonstrating how Rerun's transform system handles the spatial relationship between 3D world coordinates
and 2D image coordinates through pinhole projections.
```python
"""Demonstrates pinhole camera projections with Rerun blueprints."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_pinhole_projections", spawn=True)
img_height, img_width = 12, 16
# Create a 3D scene with a camera and an image.
rr.log(
"world/box",
rr.Boxes3D(centers=[0, 0, 0], half_sizes=[1, 1, 1], colors=[255, 0, 0]),
)
rr.log(
"world/points",
rr.Points3D(
positions=[(1, 0, 0), (-1, 0, 0), (0, 1, 0), (0, -1, 0), (0, 0, 1)],
colors=[
(255, 0, 0),
(0, 255, 0),
(0, 0, 255),
(255, 255, 0),
(255, 0, 255),
],
radii=0.1,
),
)
rr.log(
"camera",
rr.Transform3D(translation=[0, 3, 0]),
rr.Pinhole(
width=img_width,
height=img_height,
focal_length=10,
camera_xyz=rr.ViewCoordinates.LEFT_HAND_Z_UP,
),
)
# Create a simple test image.
checkerboard = np.zeros((img_height, img_width, 1), dtype=np.uint8)
checkerboard[
(np.arange(img_height)[:, None] + np.arange(img_width)) % 2 == 0
] = 255
rr.log("camera/image", rr.Image(checkerboard))
# Use a blueprint to show both 3D and 2D views side by side.
blueprint = rrb.Blueprint(
rrb.Horizontal(
# 3D view showing the scene and camera
rrb.Spatial3DView(
origin="world",
name="3D Scene",
contents=["/**"],
overrides={
# Adjust visual size of camera frustum in 3D view for
# better visibility.
"camera": rr.Pinhole.from_fields(image_plane_distance=1.0)
},
),
# 2D projection from angled camera
rrb.Spatial2DView(
# Make sure that the origin is at the camera's path.
origin="camera",
name="Camera",
contents=["/**"], # Add everything, so 3D objects get projected.
),
)
)
rr.send_blueprint(blueprint)
```
### View coordinates
You can use the [`ViewCoordinates`](https://rerun.io/docs/reference/types/archetypes/view_coordinates) archetype to set your preferred view coordinate systems, giving semantic meaning to the XYZ axes of the space.
For 3D spaces it can be used to log what the up-axis is in your coordinate system. This will help Rerun set a good default view of your 3D scene, as well as make the virtual eye interactions more natural. In Python this can be done with `rr.log("/", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True)`.
Note that in this example the archetype is logged at the root path, this will make it apply to all 3D views. Generally, a 3D view picks up view coordinates at or above its origin entity path.
[Pinholes](https://rerun.io/docs/reference/types/archetypes/view_coordinates) have a view coordinates field integrated as a shortcut.
The default coordinate system for pinhole entities is `RDF` (X=Right, Y=Down, Z=Forward).
> [!WARNING]
> Unlike in 3D views where `rr.ViewCoordinates` only impacts how the rendered scene is oriented, applying `rr.ViewCoordinates` to a pinhole-camera will actually influence the projection transform chain. Under the hood this value inserts a hidden transform that re-orients the axis of projection. Different world-content will be projected into your camera with different orientations depending on how you choose this value. See for instance the [`open_photogrammetry_format`](https://rerun.io/examples/3d-reconstruction/open_photogrammetry_format) example.
For 2D spaces and other entities, view coordinates currently have currently no effect ([#1387](https://github.com/rerun-io/rerun/issues/1387)).
### Pose transforms
[`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) defines geometric poses relative to an entity's transform frame.
Unlike with [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d), poses do not propagate through the transform hierarchy
and can store an arbitrary amount of transforms on the same entity.
For an entity that has both [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d)
(without `child_frame`/`parent_frame`) and `InstancePoses3D`,
the [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d) is applied first
(affecting the entity and all its children), then [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d)
is applied only to that specific entity.
(This is consistent with how entity hierarchy based transforms translate to transform frames.)
#### Instancing
Rerun's [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) archetype is not only used
to model poses relative to an Entity's frame, but also for repeating (known as "instancing") visualizations on the same entity:
most visualizations will show once for each transform on [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d)
in the respective place.
```python
"""
Log a simple 3D mesh with several instance pose transforms.
This instantiate the mesh several times and will not
affect its children. This is known as mesh instancing.
"""
import rerun as rr
rr.init("rerun_example_mesh3d_instancing", spawn=True)
rr.set_time("frame", sequence=0)
rr.log(
"shape",
rr.Mesh3D(
vertex_positions=[[1, 1, 1], [-1, -1, 1], [-1, 1, -1], [1, -1, -1]],
triangle_indices=[[0, 2, 1], [0, 3, 1], [0, 3, 2], [1, 3, 2]],
vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255], [255, 255, 0]],
),
)
# This box will not be affected by its parent's instance poses!
rr.log(
"shape/box",
rr.Boxes3D(half_sizes=[[5.0, 5.0, 5.0]]),
)
for i in range(100):
rr.set_time("frame", sequence=i)
rr.log(
"shape",
rr.InstancePoses3D(
translations=[[2, 0, 0], [0, 2, 0], [0, -2, 0], [-2, 0, 0]],
rotation_axis_angles=rr.RotationAxisAngle(
[0, 0, 1], rr.Angle(deg=i * 2)
),
),
)
```
In this example, the mesh at `"shape"` is instantiated four times with different translations and rotations.
The box at `"shape/box"` is not affected by its parent's instance poses and appears only once.
# Query semantics & partial updates
## The Rerun data model is based around streams of entities with components
In Rerun, you model your data using entities (roughly objects) with [batches of components](https://rerun.io/docs/concepts/logging-and-ingestion/batches.md) that change over time.
An entity is identified by an entity path, e.g. `/car/lidar/points`, where the path syntax can be used to model hierarchies of entities.
A point cloud could be made up of positions and colors, but you can add whatever components you like to the entity.
Point positions are e.g. represented as a batch of `Position3D` component instances.
Components can have different values for different times, and do not have to be updated all at once.
Rerun supports multiple timelines (sequences of times), so that you can explore your data organized according to e.g. the camera's frame index or the time it was logged.
## Core queries
All data that gets sent to the Rerun viewer is stored in an in-memory database, and there are two core types of queries against the database that visualizers in the viewer run.
**Latest-at queries** collect the latest version of each of an entity's components at a particular time.
This allows the visualizer to draw the current state of an object that was updated incrementally.
For example, you might want to update the vertex positions of a mesh while keeping textures and triangle indices constant.
**Range queries** instead collect all components associated with times on a time range.
These queries drive any visualization where data from more than one time is shown at the same time.
The obvious example is time series plots,
but it can also be used to e.g. show lidar point clouds from the last 10 frames together.
The queried range is typically configurable, see for instance [this how-to guide on fixed windows plots](https://rerun.io/docs/howto/visualization/fixed-window-plot.md) for more information.
## Partial updates
As mentioned above, the query semantics that power the Rerun Viewer, coupled with our [chunk-based storage](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md), make it possible to log only the components that have changed in-between frames (or whatever atomic unit [your timeline](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) is using).
Here's an example of updating only some specific properties of a point cloud, over time:
```python
"""Update specific properties of a point cloud over time."""
import rerun as rr
rr.init("rerun_example_points3d_partial_updates", spawn=True)
positions = [[i, 0, 0] for i in range(10)]
rr.set_time("frame", sequence=0)
rr.log("points", rr.Points3D(positions))
for i in range(10):
colors = [[20, 200, 20] if n < i else [200, 20, 20] for n in range(10)]
radii = [0.6 if n < i else 0.2 for n in range(10)]
# Update only the colors and radii, leaving everything else as-is.
rr.set_time("frame", sequence=i)
rr.log("points", rr.Points3D.from_fields(radii=radii, colors=colors))
# Update the positions and radii, and clear everything else in the process.
rr.set_time("frame", sequence=20)
rr.log(
"points",
rr.Points3D.from_fields(clear_unset=True, positions=positions, radii=0.3),
)
```
To learn more about how to use our partial updates APIs, refer to [this page](https://rerun.io/docs/howto/logging-and-ingestion/send-partial-updates.md).
# Sinks
Sinks control where your Rerun data goes. They are the output destinations for your logged data.
When you log data with Rerun, that data needs to flow somewhere, whether that's to a live viewer, a file on disk, memory, or multiple destinations at once. Sinks provide this routing layer, giving you flexible control over how and where your recordings are stored and displayed.
## Available sink types
Rerun provides several built-in sink types, each designed for specific use cases:
### GrpcSink
Streams data to a Rerun Viewer over gRPC. This is the most common sink for live visualization.
```python
"""Create and set a GRPC sink."""
import rerun as rr
rr.init("rerun_example_grpc_sink")
# The default URL is `rerun+http://127.0.0.1:9876/proxy`
# This can be used to connect to a viewer on a different machine
rr.set_sinks(rr.GrpcSink("rerun+http://127.0.0.1:9876/proxy"))
```
### FileSink
Writes data to `.rrd` files on disk.
```python
"""Create and set a file sink."""
import rerun as rr
rr.init("rerun_example_file_sink")
rr.set_sinks(rr.FileSink("recording.rrd"))
```
## Multiple sinks (Tee pattern)
One of the most powerful features of Rerun's sink system is the ability to send data to multiple destinations simultaneously. This "tee" pattern lets you both visualize data live and save it to disk in a single run.
```python
"""Log some data to a file and a Viewer at the same time."""
import numpy as np
import rerun as rr
# Initialize the SDK and give our recording a unique name
rr.init("rerun_example_set_sinks")
rr.set_sinks(
# Connect to a local viewer using the default URL
rr.GrpcSink(),
# Write data to a `data.rrd` file in the current directory
rr.FileSink("data.rrd"),
)
# Create some data
SIZE = 10
pos_grid = np.meshgrid(*[np.linspace(-10, 10, SIZE)] * 3)
positions = np.vstack([d.reshape(-1) for d in pos_grid]).T
col_grid = np.meshgrid(*[np.linspace(0, 255, SIZE)] * 3)
colors = np.vstack([c.reshape(-1) for c in col_grid]).astype(np.uint8).T
# Log the data
rr.log(
# name under which this entity is logged (known as "entity path")
"my_points",
# log data as a 3D point cloud archetype
rr.Points3D(positions, colors=colors, radii=0.5),
)
```
This pattern is useful when:
- You want to monitor a long-running process while archiving the data
- You're debugging and want both live feedback and a recording to analyze later
- You need to stream to multiple viewers or save to multiple files
## See also
- [Recordings](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md): Understand how recordings relate to sinks
- [Blueprints](https://rerun.io/docs/concepts/visualization/blueprints.md): Learn how to configure the viewer's layout
- API References:
- [π Python sinks API](https://ref.rerun.io/docs/python/stable/common/initialization_functions/)
- [π¦ Rust RecordingStream](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html)
- [π C++ RecordingStream](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html)
# MCAP files
Working with MCAP files in Rerun:
* [Working with MCAP](https://rerun.io/docs/howto/logging-and-ingestion/mcap.md)
* [Supported Message Formats](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md)
Technical details and advanced usage:
* [MCAP Decoders Explained](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/decoders-explained.md)
* [CLI Reference for MCAP](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/cli-reference.md)
# Recordings
## What is a recording?
In their simplest form, recordings can be thought of as individual `.rrd` files containing Rerun data (or, equivalently, a single [stream](https://rerun.io/docs/concepts/how-does-rerun-work.md) of data generated by a logging process).
In practice, what we call recording is more nuanced, sometimes depends on context, and can take many shapes.
This page aims to explain this in detail.
### Logical vs physical recordings
The recording/file analogy comes short of describing how the Rerun Viewer handles data.
When the Viewer receives data, whether by loading a `.rrd` file or an incoming logging stream, it pools the corresponding data by recording ID and application ID.
This can be thought of as a logical recording, even though its source might be multiple files.
This implicit merging semantics also implies that, from the perspective of the Viewer, recordings are never "completed."
This enables the [distributed logging workflows](#distributed-recordings) described below.
In its UI, the Viewer presents (logical) recordings sharing the same application ID as related.
In particular, they share the same [blueprint](https://rerun.io/docs/concepts/visualization/blueprints.md).
### Recordings on a catalog server
A catalog server has a slightly different object model, which you can read more about in [Catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md).
Datasets are top-level objects that group semantically related episodes of data, which we call _segments_.
For example, it can be multiple recordings of the same robotic task.
Blueprints can optionally be assigned to datasets, so all segments in a dataset share the same blueprint.
Populating a dataset happens by registering recordings using the Catalog SDK.
Its recording ID becomes the segment ID, and its application ID is discarded.
Segments can contain multiple _layers_ identified by their name, each backed by a `.rrd` file.
This again allows pooling multiple physical recordings into a single (logical) segment.
### Distributed recordings
Both the Viewer's implicit merging semantics and the catalog server's layer system enable distributed logging workflows. Multiple processes or machines can produce separate `.rrd` files that share the same recording ID and application ID.
When these files are loaded into the Viewer, they are treated as a single logical recording. Alternatively, when using a catalog server, these files can be registered to separate layers. This enables workflows where data collection is distributed across multiple sources but visualized as a unified set of data.
You can learn more about this in the [shared recordings guide](https://rerun.io/docs/howto/logging-and-ingestion/shared-recordings.md).
### Storage formats
Rerun recordings are stored in `.rrd` files. [Blueprints](https://rerun.io/docs/concepts/visualization/blueprints.md) are also recordings, albeit ones containing layout information instead of data. By convention, the `.rbl` file extension is used for blueprints.
## Application IDs
Rerun recordings have an _application ID_ in their metadata.
Application IDs are arbitrary user-defined strings set when initializing the SDK:
```python
rr.init("my_custom_application_id")
```
### When application IDs matter
Application IDs are used by the Viewer when loading recordings directly (not via a catalog server):
- The Viewer stores blueprints per application ID
- Different recordings share the same blueprint if they share the same application ID
- Recordings are grouped by application ID in the Viewer UI
As stated above, application IDs are discarded when registering recordings to a catalog server. See [Recordings on a catalog server](#recordings-on-a-catalog-server) above.
Check out the API to learn more about SDK initialization:
- [π Python](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.init)
- [π¦ Rust](https://docs.rs/rerun/latest/rerun/struct.RecordingStreamBuilder.html#method.new)
- [π C++](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#abda6202900fa439fe5c27f7aa0d1105a)
## Recording IDs
By default, a random recording ID is generated each time you start logging.
This means that, by default, separate logging sessions will produce separate (logical) recording when loaded in the Viewer, and separate segments when registered to a dataset.
You can override the default recording ID when initializing the SDK (or the recording stream):
```python
rr.init("rerun_example_shared_recording", recording_id="my_shared_recording")
```
This enables the distributed logging workflow described above, as well as assigning specific segment ID for recordings to be registered to datasets.
# Chunk Processing API
The Chunk Processing API is a flexible, [chunk](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md)-centric API for data ingestion, transformation, and conversion pipelines.
It covers I/O from common robotics file formats, powerful declarative data wrangling primitives, and a multithreaded, native engine for pipeline execution.
The API is designed to support distributed execution in the future.
> [!NOTE]
> The Chunk Processing API is currently experimental and may change in future releases. It is available in the Python SDK under `rerun.experimental`.
## Building blocks
The Chunk Processing API is built from three kinds of primitives β readers, stores, and lazy streams β that compose into a pipeline executed by a terminal call:
### Readers
Readers produce [`Chunk`](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md)s from external sources such as files, or datasets hosted on a catalog server.
In some cases, readers are classes provided by the Chunk Processing API, such as [`RrdReader`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.RrdReader) and [`McapReader`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.McapReader).
The reader functionality can also be provided by classes from other parts of the Rerun SDK.
For example, [`DatasetEntry`](https://ref.rerun.io/docs/python/stable/catalog/#rerun.catalog.DatasetEntry) has a [`segment_store`](https://ref.rerun.io/docs/python/stable/catalog/#rerun.catalog.DatasetEntry.segment_store) method which returns a [`LazyStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyStore) for the corresponding segment (see the [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) for more information on datasets).
[`UrdfTree`](https://ref.rerun.io/docs/python/stable/urdf/#rerun.urdf.UrdfTree) is another example of a class that offers reader functionality in addition to a larger feature set.
There are two ways in which a reader may provide chunks.
All readers can sequentially stream all their source's chunks, typically via the `stream()` method.
Internally, such readers typically parse the source file, convert data to chunks as it is extracted, and yield those chunks as they are produced.
Some readers, called [`IndexedReader`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.IndexedReader), can also provide indexed, random access to chunks via a [`LazyStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyStore).
This is typically implemented on top of an existing chunk index, and is currently available for the following readers:
- `DatasetEntry.segment_store()` (relies on the chunk index maintained by the catalog server)
Processing chunks through a `LazyStore` is beneficial for pipelines where only a subset of chunks is needed, avoiding the I/O cost of loading unnecessary ones.
> [!NOTE]
> Filter pushdown to `LazyStore` (e.g. `lazy_store.stream().filter(content="/my/entity")`) is planned but not yet implemented; today the filter runs after the chunks have been loaded.
In all cases, readers typically act as the root of a processing pipeline and provide a `LazyChunkStream` object to refine and execute it β see [Lazy stream](#lazy-stream) below.
### Stores
A store is a collection of chunks and comes in two complementary flavors:
- **[`LazyStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyStore)** β index-based, on-demand. Returned by indexed loaders such as `RrdReader(path).store()` and `DatasetEntry.segment_store()`.
- **[`ChunkStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.ChunkStore)** β fully materialized, all chunks held in memory. Build one with `ChunkStore.from_chunks([...])`, or materialize a stream via `stream.collect()`.
The previous section already hinted at the perks of `LazyStore`. Being index-based, it is cheap to create and takes limited amounts of memory.
Also, it unlocks performance speed-ups by only loading chunks that are relevant to the given processing pipeline.
On the other hand, `ChunkStore` is fully materialized: its memory footprint scales with the recording size.
This is a major exception in the chunk processing API, which generally leans on lazy loading and streaming execution to allow processing large datasets with bounded memory.
Both kinds of stores share a common API surface, including:
- extracting the underlying [`Schema`](https://ref.rerun.io/docs/python/stable/catalog/#rerun.catalog.Schema) of the store;
- turning the store back into a pipeline with `.stream()`;
- exposing various statistics and content summaries.
One common reason to materialize a `ChunkStore` is to run chunk optimization; see [Optimize chunk count](https://rerun.io/docs/howto/logging-and-ingestion/optimize-chunks.md) for details.
A materialized `ChunkStore` can also be queried directly as a dataframe with [`ChunkStore.reader`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.ChunkStore.reader), without spinning up a catalog server.
The returned [DataFusion](https://datafusion.apache.org/) dataframe is data-equivalent to loading the same chunks into a dataset and calling its `reader()`, so the full [dataframe query API](https://rerun.io/docs/concepts/query-and-transform/dataframe-queries.md) applies β modulo the `rerun_segment_id` column (see the [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md) for more information about datasets and segments).
For example, first materialize a store (here built a single chunk, for illustration):
```python
import rerun as rr
from rerun.experimental import Chunk, ChunkStore
chunk = Chunk.from_columns(
"/sensor",
indexes=[rr.TimeColumn("frame", sequence=[0, 1, 2, 3])],
columns=rr.Scalars.columns(scalars=[0.0, 0.5, 1.0, 1.5]),
)
store = ChunkStore.from_chunks([chunk])
```
Then the `ChunkStore` can be queried directly:
```python
df = store.reader(index="frame")
df = df.filter(col("/sensor:Scalars:scalars")[0] >= 1.0)
print(df) # or convert to Pandas, Polars, PyArrow, etc.
```
### Lazy stream
The `LazyChunkStream` is the central abstraction: a deferred, single-pass iterator of chunks with operators for filtering (`filter` / `drop`), branching (`split`), fan-in (`merge`), reshaping (`lenses`), and arbitrary per-chunk manipulation (`map` / `flat_map`).
The key design is that a lazy stream is not a materialized collection or actual streaming process.
A `LazyChunkStream` instance can be thought of as a leaf node in a pipeline-description [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph).
By composition, it allows building up the DAG to represent the intended pipeline.
For example, this creates a basic pipeline that does nothing but read an MCAP file:
```python
from rerun.experimental import McapReader
stream = McapReader(mcap_path).stream()
```
This pipeline can be extended using the lazy stream's methods.
For example, we can add a filter operation:
```python
stream = stream.filter(content="/robot_left/**")
```
Up to this point, no data has actually been read or processed.
This happens when a terminal operation is called, for example:
```python
stream.write_rrd(
output_path,
application_id="rerun_example_chunk_processing_intro",
recording_id="run1",
)
```
This exact call triggers the pipeline execution, including reading the source MCAP, performing the filter operation, and writing the output RRD.
#### Pipeline execution
To recap:
- A pipeline is a DAG rooted at one or more readers or stores and ending at a leaf node represented by a lazy stream.
- Composition is cheap: building the DAG is metadata only, regardless of input size. This is done through `LazyChunkStream`'s APIs.
- The actual execution of the pipeline is triggered by calling a terminal method of the lazy stream, for example `.write_rrd()`. Terminal calls are blocking, but execution is multithreaded and essentially GIL-free.
- Memory cost is bounded by what flows through a chunk at a time, not by the total recording size.
#### Move semantics
To better express the DAG composition process, `LazyChunkStream` instances exhibit Rust-like move semantics to avoid accidental reuse:
- `stream.filter(...)` moves `stream` into the new pipeline. Reusing `stream` afterwards raises `ValueError: already been consumed`.
- `stream.split(...)` returns two branches and consumes the parent. Each branch is itself a stream that can only be consumed once.
- `LazyChunkStream.merge(a, b, ...)` consumes every input.
Terminal calls, however, do not consume the stream β a lazy stream can be executed multiple times against different destinations:
```python
chunk_list = stream.to_chunks()
stream.write_rrd(path=..., application_id=..., recording_id=...)
```
Note that doing so executes the entire pipeline twice, which may not be desirable for complex pipelines. In that case, collect the stream to an intermediate `ChunkStore` to trade memory for re-computation.
## Complete example
The rest of this page walks through a single end-to-end pipeline that reads a robot-arm MCAP recording, fans the protobuf joint-state column out into per-joint `Scalars` series in degrees, tags the result with a static `/metadata` chunk built from scratch, and writes a new `.rrd`.
Full source: [Python](https://github.com/rerun-io/rerun/blob/main/docs/snippets/all/concepts/chunk_processing.py).
### Setup
```python
from __future__ import annotations
import math
import uuid
from collections.abc import Callable
from pathlib import Path
import pyarrow as pa
import pyarrow.compute as pc
import rerun as rr
from rerun.experimental import (
Chunk,
DeriveLens,
LazyChunkStream,
McapReader,
Selector,
)
MCAP = (
Path(__file__).resolve().parents[4]
/ "tests"
/ "assets"
/ "mcap"
/ "trossen_transfer_cube.mcap"
)
OUT = Path("chunk_processing.rrd")
```
- Imports the experimental entry points: readers (`McapReader`), chunk and stream types (`Chunk`, `LazyChunkStream`), lens primitives (`DeriveLens`, `Selector`).
- Locates the input MCAP relative to the repo root and picks a CWD-relative output path. Nothing here touches Rerun yet.
### Reading
```python
stream = McapReader(MCAP).stream()
```
- `McapReader(MCAP).stream()` is the only line that touches the source β and even that is lazy: no MCAP bytes are decoded yet.
- The returned `LazyChunkStream` is the root of the DAG.
### Processing
```python
JOINTS = [
"waist",
"shoulder",
"elbow",
"forearm_roll",
"wrist_angle",
"wrist_rotate",
]
def pick_joint(i: int) -> Callable[[pa.Array], pa.Array]:
"""Extract joint `i` from a list column and convert rad β deg."""
return lambda arr: pc.multiply(pc.list_element(arr, i), 180.0 / math.pi)
def fan(side: str) -> list[DeriveLens]:
return [
DeriveLens(
"schemas.proto.JointState:message",
output_entity=f"/joints_deg/{side}/{name}",
).to_component(
rr.Scalars.descriptor_scalars(),
Selector(".joint_positions").pipe(pick_joint(i)),
)
for i, name in enumerate(JOINTS)
]
processed = (
stream
.drop(content="/video_raw/**")
.lenses(
fan("left"),
content="/robot_left/**",
output_mode="forward_unmatched",
)
.lenses(
fan("right"),
content="/robot_right/**",
output_mode="forward_unmatched",
)
)
```
- `drop(content="/video_raw/**")` is a no-op against this MCAP (the path does not exist) but illustrates content-based pruning.
- `fan(side)` builds six `DeriveLens` instances, one per joint, each extracting `.joint_positions[i]` (via `Selector(...).pipe(...)`), converting radians to degrees with `pyarrow.compute`, and routing the result to `/joints_deg//` as a `Scalars` column.
- Two scoped `.lenses(...)` calls apply the per-side fan only to chunks under `/robot_left/**` and `/robot_right/**` respectively. The same component name (`schemas.proto.JointState:message`) lives on both sides; scoping by `content=` is what disambiguates them. With `forward_unmatched`, every chunk outside the scope passes through untouched.
### Merging
```python
metadata = Chunk.from_columns(
"/metadata",
indexes=[],
columns=rr.AnyValues.columns(
processing_type="ingestion",
processing_version="v1",
),
)
merged = LazyChunkStream.merge(processed, LazyChunkStream.from_iter([metadata]))
```
- `Chunk.from_columns("/metadata", indexes=[], columns=rr.AnyValues.columns(β¦))` builds a single static chunk from scratch β `indexes=[]` makes it static. Any archetype's `.columns(β¦)` helper works here.
- `LazyChunkStream.from_iter([metadata])` lifts that one chunk into a one-element stream so it can participate in the pipeline.
- `LazyChunkStream.merge(processed, ...)` is fan-in: the two inputs become one stream. Order is preserved per-input, not globally.
### Writing
```python
merged.write_rrd(
OUT,
application_id="rerun_example_chunk_processing",
recording_id=str(uuid.uuid4()),
)
```
- `write_rrd(...)` is the terminal: this is where the DAG actually executes. The whole pipeline runs in a single streaming pass.
- `application_id` and `recording_id` identify the resulting recording; a fresh `uuid.uuid4()` makes each invocation produce a distinct recording.
## Relationship to the logging APIs
Both the logging APIs (`rr.log`, `rr.send_columns`, `RecordingStream`) and the Chunk Processing API target the same underlying data model, but they differ in several ways:
| | Logging API | Chunk processing API |
|------------------------|------------------------------------------|---------------------------------------------------------------------------------------------------------|
| Direction | logging call β sink | chunk source β transform β chunk sink |
| Granularity | single rows or columns of data | whole chunks |
| Execution model | continuous, as logging calls are emitted | lazy, upon stream execution |
| Where chunks come from | built by the logging API's batcher | already exist (from a reader) or built explicitly with `Chunk.from_columns` / `Chunk.from_record_batch` |
| Typical use | realtime data logging | ingestion, conversion, post-processing pipelines |
The two are interoperable:
- **Logging β chunk processing:** save a `RecordingStream` to an `.rrd`, then re-open it with `RrdReader` to get a `LazyChunkStream`.
> [!NOTE]
> This roundtrip-via-file will be smoothed out in the future for better ergonomics and performance.
- **Chunk processing β logging:** `rerun.experimental.send_chunks(chunks, recording=...)` feeds chunks into an active `RecordingStream` (useful for streaming to a viewer, for example).
- **Building chunks by hand:** `Chunk.from_columns` mirrors `rr.send_columns` and accepts the same `rr..columns(...)` helpers, so any data that can be logged with `rr.send_columns` can also be packaged as a `Chunk` and injected into a processing pipeline.
Likewise, `Chunk.from_record_batch` (for a single `RecordBatch`) and `Chunk.from_dataframe` (a multi-batch `Table`, `RecordBatchReader`, or `datafusion.DataFrame`) mirrors `rr.send_record_batch` and `rr.send_dataframe`.
See [Chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) for details.
## See also
- [Chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md): the underlying data model.
- [Lenses](https://rerun.io/docs/concepts/query-and-transform/lenses.md): the reshaping primitives used here.
- [`robot_data_preprocessing`](https://github.com/rerun-io/rerun/tree/main/examples/python/robot_data_preprocessing): a practical example showing how to apply the chunk processing API to robot data.
# RRD format
An RRD is the file format Rerun uses to persist recordings and blueprints. At the lowest level it is a linear sequence of framed messages β store announcements and chunks of data β optionally followed by a footer index that makes random access cheap. This page covers the envelope around chunks and how they are serialized; the chunk data model itself is described in [Chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md).
## Stores
Logical groupings of chunks form so-called stores.
They come in two flavors: [recording](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) and [blueprint](https://rerun.io/docs/concepts/visualization/blueprints.md).
Both are structurally identical and distinguished only by a flag (store kind).
A single RRD can hold any number of stores.
The file extension is either `.rrd` or `.rbl`.
Both refer to the exact same on-disk format and are used conventionally:
- `.rrd` files hold any combination of recording and blueprint stores;
- `.rbl` files hold a single blueprint store.
## Message kinds (`LogMsg`)
The body of an RRD is a sequence of `LogMsg`s. There are three variants:
- **`SetStoreInfo`** announces a new store and carries its [`StoreInfo`](#store-metadata-storeinfo).
It must appear before any data for that store.
There can be more than one `SetStoreInfo` for the same store in a single stream β for example, when a `RecordingStream` is created and later attached to a `FileSink` β and the latest one wins.
- **`ArrowMsg`** carries the actual data: an [Apache Arrow IPC](https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format) payload encoding a single chunk, tagged with the `StoreId` it belongs to.
This is what makes up the bulk of every RRD.
- **`BlueprintActivationCommand`** is the only non-data control message.
It is emitted after a blueprint's chunks have been sent, and lets the producer atomically activate the blueprint via the [`make_active` / `make_default`](https://ref.rerun.io/docs/python/stable/blueprint/) flags.
It exists so that the Viewer never sees a half-loaded blueprint, and so the application can decide whether to apply the blueprint as the current one or the default.
> [!NOTE]
> At the wire level there is also an `End` message kind that frames the optional footer described [below](#footer). It is not a `LogMsg` variant in the application-level type system β it is an envelope reserved for the footer payload β but it shares the same framing as the three `LogMsg`s above.
## Chunks (`ArrowMsg` payload)
Every `ArrowMsg` carries a single **chunk** β an Apache Arrow `RecordBatch` with Rerun-specific schema metadata. A chunk belongs to one entity path and holds a contiguous run of rows for that entity, with one column per timeline and one column per component. See [Chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) for the conceptual deep-dive (how chunks are built, batched, sorted, compacted); this section just shows what a chunk looks like when you crack one open.
The schema is laid out per **Sorbet**, Rerun's object-model spec β it defines how chunks, archetypes, components, and timelines map onto Arrow column names, types, and metadata. The easiest way to see it concretely is to save a recording and reopen it with [`RrdReader`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.RrdReader).
First, let's create an RRD file with some content:
```python
with rr.RecordingStream(
"rerun_example_rrd_format", recording_id="example"
) as rec:
rec.save(output_path)
rec.set_time("frame", sequence=0)
rec.log(
"/points",
rr.Points3D(
[[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]],
colors=[(255, 0, 0), (0, 255, 0)],
),
)
rec.set_time("frame", sequence=1)
rec.log("/points", rr.Points3D([[2.0, 2.0, 2.0]], colors=[(0, 0, 255)]))
```
Then we can inspect the first chunk it contains:
```python
from rerun.experimental import RrdReader
reader = RrdReader(output_path)
for chunk in reader.stream():
if chunk.entity_path == "/points":
print(chunk.format(trim_metadata_keys=False))
break
```
> [!NOTE]
> By default, `chunk.format()` trims metadata keys to keep the representation concise.
> Using `trim_metadata_keys=False` disables this behavior, so the typical `rerun:` / `sorbet:` prefixes are visible here.
This prints a chunk together with its schema. A typical output looks like:
```text
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β METADATA: β
β * rerun:entity_path: /points β
β * rerun:id: chunk_18B0AA9FA7B7B1A61d23c55ca87b18b4 β
β * sorbet:version: 0.1.3 β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β βββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββββ β
β β RowId β frame β log_tick β log_time β Points3D:colors β Points3D:positions β β
β β --- β --- β --- β --- β --- β --- β β
β β type: non-null FixedSizeBinary(16) β type: Int64 β type: Int64 β type: Timestamp(ns) β type: List(UInt32) β type: List(FixedSizeList(3 x β β
β β ARROW:extension:metadata: β rerun:index_name: frame β rerun:index_name: log_tick β rerun:index_name: log_time β rerun:archetype: Points3D β non-null Float32)) β β
β β {"namespace":"row"} β rerun:is_sorted: true β rerun:is_sorted: true β rerun:is_sorted: true β rerun:component: Points3D:colors β rerun:archetype: Points3D β β
β β ARROW:extension:name: TUID β rerun:kind: index β rerun:kind: index β rerun:kind: index β rerun:component_type: Color β rerun:component: Points3D:positions β β
β β rerun:is_sorted: true β β β β rerun:kind: data β rerun:component_type: Position3D β β
β β rerun:kind: control β β β β β rerun:kind: data β β
β βββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββββ‘ β
β β row_18B0AA9FA79D51886952b7c6bb9f6ed β 0 β 0 β 2026-05-18T13:04:15.500740 β [4278190335, 16711935] β [[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]] β β
β β 4 β β β β β β β
β βββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββββ€ β
β β row_18B0AA9FA7B6D7E06952b7c6bb9f6ed β 1 β 1 β 2026-05-18T13:04:15.501658 β [65535] β [[2.0, 2.0, 2.0]] β β
β β 5 β β β β β β β
β βββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββββ β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
What to notice:
- All Rerun-specific metadata keys are prefixed with `rerun:` (`rerun:entity_path`, `rerun:id`, `rerun:kind`, `rerun:index_name`, β¦). Sorbet's own metadata uses the `sorbet:` prefix (`sorbet:version`).
- The chunk-level **metadata** identifies the entity path the chunk belongs to and the chunk's id.
- The **`RowId`** column is the row identity column (`rerun:kind: control`).
- Each timeline contributes one **index column** (`frame`, `log_tick`, `log_time`) β `log_time` is auto-populated by the logging API (and `log_tick` if opted in), `frame` is the user-defined timeline.
- Each component contributes one **data column** (`Points3D:colors`, `Points3D:positions`) carrying the per-row values.
## Store metadata (`StoreInfo`)
Every store in an RRD is identified by a `StoreId` and described by a `StoreInfo`:
- **`StoreId`** combines:
- **`kind`** β `Recording` or `Blueprint`. The on-disk format treats both identically; the kind is just a flag. What differs is the *expected content*: recordings hold user-logged data on user-defined entity paths, blueprints hold `rr.blueprint.*` objects on Viewer-reserved paths. The Viewer dispatches on the kind β recordings populate the data store, blueprints populate the Viewer's UI/layout state.
- **`application_id`** β a user-chosen identifier for the application that produced the recording (see [Recordings](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) for the conventions, including the relationship with segment and dataset IDs in the remote/catalog context).
- **`recording_id`** β a UUID or user-chosen string that distinguishes runs of the same application (catalog servers use this as the segment ID β see the [catalog object model](https://rerun.io/docs/concepts/query-and-transform/catalog-object-model.md)).
- **`StoreInfo`** wraps the `StoreId` and adds:
- **`cloned_from`** β for stores that originated as a clone of another (typically the active blueprint is derived from a default blueprint).
- **`store_source`** β where the store came from (`PythonSdk`, `RustSdk`, `CppSdk`, or a file source such as CLI / drag-drop).
- **`store_version`** β the Rerun version that produced the data.
Matching `application_id` and `recording_id` is how the Viewer merges multiple `.rrd` files (or multiple stores within one file) into a single logical recording.
`.rbl` is just an RRD whose store happens to have `kind = Blueprint` β nothing in the bytes makes it special.
The convention of using `.rbl` for blueprints instead of `.rrd` is purely a filename hint to the Viewer and users.
When an RRD holds multiple [stores](#stores) each store begins with its own `SetStoreInfo`, and every subsequent `ArrowMsg` is tagged with its store's `StoreId`.
Messages from different stores may be interleaved or grouped.
The [footer](#footer) indexes each store separately, so readers can enumerate stores and select the ones they want without scanning chunk bytes.
## Footer
The footer is an optional manifest appended at the end of an RRD that enables random access into the file.
For each chunk in the RRD, the manifest carries chunk-level metadata (id, byte offset in the file, byte size β compressed and uncompressed) along with per-component and per-timeline statistics and the chunk's schema hash.
Like all data in Rerun, the manifest is internally stored as an Arrow `RecordBatch`, with one row per chunk.
With the footer, a reader can enumerate stores in a handful of seeks and pull only the chunks it actually needs β for example, by entity path or by time range β without reading any chunk it does not care about.
This is what enables [`RrdReader`](https://rerun.io/docs/concepts/logging-and-ingestion/chunk-processing-api.md) to be cheap to use on large files, and the OSS catalog server to "load" large datasets quickly and with little memory overhead.
All tooling included in recent versions of the Rerun SDK emit footers by default.
An RRD may still miss a footer for a variety of reasons β for example, when a stream is not shut down cleanly, or legacy RRDs written before footers existed.
In those cases, readers fall back to a linear scan, which is semantically equivalent β just slower for partial reads.
For illustration, let's see what a footer looks like in an RRD.
This can be done with the following command:
```sh
rerun rrd print --footers --footers-lod 2 my.rrd
```
Here we use `--footers-lod 2` to see the entire table, which happen to be very wide. Here is the result for the recording produced by the snippet above:
```text
Showing data after migration to latest Rerun version
StoreInfo {
store_id: StoreId(
Recording,
"rerun_example_rrd_format",
"example",
),
cloned_from: None,
store_source: PythonSdk(
3.11.13,
),
store_version: Some(
CrateVersion {
major: 0,
minor: 33,
patch: 0,
meta: Some(
DevAlpha {
alpha: 1,
commit: None,
},
),
},
),
}
StoreInfo {
store_id: StoreId(
Recording,
"rerun_example_rrd_format",
"example",
),
cloned_from: None,
store_source: PythonSdk(
3.11.13,
),
store_version: Some(
CrateVersion {
major: 0,
minor: 33,
patch: 0,
meta: Some(
DevAlpha {
alpha: 1,
commit: None,
},
),
},
),
}
Chunk(chunk_18B0AA9F967A41276952b7c6bb9f6ed2) with 1 rows (632 B) - /__properties - data columns: [RecordingInfo:start_time]
Chunk(chunk_18B0AA9FA7B7B1A61d23c55ca87b18b4) with 2 rows (1.2 KiB) - /points - data columns: [Points3D:colors Points3D:positions]
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β METADATA: β
β * source: "/tmp/rrd_format_doc.rrd" β
β * schema_sha_256: 03bea0095483cf5d32a3d28fc28f0433d917cdeacf88a07fcf02b97a778f492b β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β βββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββ¬ββββββββββββββββββββββββ¬ββββββββββββββββββββββββ¬ββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββ¬βββββββββββββββ¬ββββββββββββββββββ¬ββββββββββββββββββ¬βββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ β
β β chunk_entity_path β chunk_id β chunk_is_static β chunk_num_rows β chunk_byte_offset β chunk_byte_size β chunk_byte_size_uncompressed β Points3D:colors:has_static_data β Points3D:positions:has_static_data β RecordingInfo:start_time:has_static_data β frame:start β frame:end β log_tick:start β log_tick:end β log_time:start β log_time:end β frame:Points3D:colors:start β frame:Points3D:colors:end β frame:Points3D:colors:num_rows β frame:Points3D:positions:start β frame:Points3D:positions:end β frame:Points3D:positions:num_rows β log_tick:Points3D:colors:start β log_tick:Points3D:colors:end β log_tick:Points3D:colors:num_rows β log_tick:Points3D:positions:start β log_tick:Points3D:positions:end β log_tick:Points3D:positions:num_rows β log_time:Points3D:colors:start β log_time:Points3D:colors:end β log_time:Points3D:colors:num_rows β log_time:Points3D:positions:start β log_time:Points3D:positions:end β log_time:Points3D:positions:num_rows β β
β β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β --- β β
β β type: non-null Utf8 β type: non-null FixedSizeBinary(16) β type: non-null Boolean β type: non-null UInt64 β type: non-null UInt64 β type: non-null UInt64 β type: non-null UInt64 β type: non-null Boolean β type: non-null Boolean β type: non-null Boolean β type: Int64 β type: Int64 β type: Int64 β type: Int64 β type: Timestamp(ns) β type: Timestamp(ns) β type: Int64 β type: Int64 β type: UInt64 β type: Int64 β type: Int64 β type: UInt64 β type: Int64 β type: Int64 β type: UInt64 β type: Int64 β type: Int64 β type: UInt64 β type: Timestamp(ns) β type: Timestamp(ns) β type: UInt64 β type: Timestamp(ns) β type: Timestamp(ns) β type: UInt64 β β
β β β β β β β β β archetype: Points3D β archetype: Points3D β archetype: RecordingInfo β index: frame β index: frame β index: log_tick β index: log_tick β index: log_time β index: log_time β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β archetype: Points3D β β
β β β β β β β β β component: Points3D:colors β component: Points3D:positions β component: RecordingInfo:start_time β β β β β β β component: Points3D:colors β component: Points3D:colors β component: Points3D:colors β component: Points3D:positions β component: Points3D:positions β component: Points3D:positions β component: Points3D:colors β component: Points3D:colors β component: Points3D:colors β component: Points3D:positions β component: Points3D:positions β component: Points3D:positions β component: Points3D:colors β component: Points3D:colors β component: Points3D:colors β component: Points3D:positions β component: Points3D:positions β component: Points3D:positions β β
β β β β β β β β β component_type: Color β component_type: Position3D β component_type: Timestamp β β β β β β β component_type: Color β component_type: Color β component_type: Color β component_type: Position3D β component_type: Position3D β component_type: Position3D β component_type: Color β component_type: Color β component_type: Color β component_type: Position3D β component_type: Position3D β component_type: Position3D β component_type: Color β component_type: Color β component_type: Color β component_type: Position3D β component_type: Position3D β component_type: Position3D β β
β β β β β β β β β index: rerun:static β index: rerun:static β index: rerun:static β β β β β β β index: frame β index: frame β index: frame β index: frame β index: frame β index: frame β index: log_tick β index: log_tick β index: log_tick β index: log_tick β index: log_tick β index: log_tick β index: log_time β index: log_time β index: log_time β index: log_time β index: log_time β index: log_time β β
β βββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββͺββββββββββββββββββββββββͺββββββββββββββββββββββββͺββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββββββββͺβββββββββββββββͺβββββββββββββββͺββββββββββββββββββͺββββββββββββββββββͺβββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββββ‘ β
β β /__properties β 18b0aa9f967a41276952b7c6bb9f6ed2 β true β 1 β 240 β 986 β 1736 β false β false β true β null β null β null β null β null β null β null β null β 0 β null β null β 0 β null β null β 0 β null β null β 0 β null β null β 0 β null β null β 0 β β
β βββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββΌββββββββββββββββββββββββΌββββββββββββββββββββββββΌββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββΌβββββββββββββββΌβββββββββββββββΌββββββββββββββββββΌββββββββββββββββββΌβββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββ€ β
β β /points β 18b0aa9fa7b7b1a61d23c55ca87b18b4 β false β 2 β 1242 β 1564 β 3656 β false β false β false β 0 β 1 β 0 β 1 β 2026-05-18T13:04:15.500740 β 2026-05-18T13:04:15.501658 β 0 β 1 β 2 β 0 β 1 β 2 β 0 β 1 β 2 β 0 β 1 β 2 β 2026-05-18T13:04:15.500740 β 2026-05-18T13:04:15.501658 β 2 β 2026-05-18T13:04:15.500740 β 2026-05-18T13:04:15.501658 β 2 β β
β βββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββ΄ββββββββββββββββββββββββ΄ββββββββββββββββββββββββ΄ββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββ΄βββββββββββββββ΄ββββββββββββββββββ΄ββββββββββββββββββ΄βββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββββββββ΄βββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
The first lines decode the `SetStoreInfo` messages and the `ArrowMsg` payloads found in the stream.
The wide table that follows is the manifest itself, with one row per chunk.
The chunk-level columns (entity path, id, sortedness, row count, byte span, uncompressed size) are followed by per-component and per-timeline statistics β global timeline ranges (`frame:start`/`end`, `log_tick:*`, `log_time:*`) and per-component-per-timeline statistics (`frame:Points3D:positions:start`/`end`/`num_rows`, β¦) β that let an indexed reader skip components within a timeline range without reading their payloads.
## File layout
This section gives a byte-level walkthrough of the framing. All multibyte integers are little-endian.
The high-level shape of any RRD is:
```
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β StreamHeader 12 bytes β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β Messageβ : MessageHeader (16 B) + payload (Nβ B) β
β Messageβ : MessageHeader (16 B) + payload (Nβ B) β
β β¦ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β End msg : MessageHeader (16 B) + RrdFooter payload β β
β β β optional
β StreamFooter 32 bytes (typ.) β β footer
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
```
The three building blocks are detailed below.
### Stream header
Every RRD opens with the same fixed 12 bytes:
```
StreamHeader β 12 bytes
ββββββββββββββ¬βββββββββββββ¬ββββββββββββββββββββββ
β FourCC β Version β EncodingOptions β
β 4 bytes β 4 bytes β 4 bytes β
ββββββββββββββ΄βββββββββββββ΄ββββββββββββββββββββββ
0 4 8 12
```
| offset | field | value |
|------------|-------------|-------------------------------------------------------------|
| `[0..4)` | FourCC | `b"RRF2"` for the current format |
| `[4..8)` | Version | 4-byte encoded Rerun crate version |
| `[8]` | compression | `0` = Off, `1` = LZ4 |
| `[9]` | serializer | `2` = Protobuf (`1` once meant MsgPack and is now rejected) |
| `[10..12)` | reserved | `0x00 0x00` |
- Older `"RRF0"` / `"RRF1"` FourCCs are recognized but rejected with `OldRrdVersion` β there is no in-place reader for them, you have to migrate through an older Rerun release.
- The historical bit-pattern `[0, 0, 0, 0]` for `Version` is interpreted as `0.2.0` (pre-2023-02-27 files); any encoded version older than `0.23` is rejected outright.
- `EncodingOptions` describes how the *payloads* of subsequent messages are encoded. In practice these flags are mostly advisory today β the values that matter ride alongside each individual message β but the bytes are still part of the format and the two reserved bytes must be zero.
> [!NOTE]
> The header format exposes legacy details that are no longer supported and may require an older Rerun SDK version to migrate.
> However, RRDs created by Rerun SDK 0.23 and later are guaranteed to be migrated, and this guarantee holds for future releases β see the next section.
### Message framing
After the header, the file is a sequence of framed messages. Each message is a 16-byte header followed by an opaque payload:
```
MessageHeader β 16 bytes
βββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββββ
β kind β payload_len β
β u64 LE β u64 LE β
βββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββββ
0 8 16
ββββββββββββββββββββββββββββββββββ
β payload β payload_len bytes β
β protobuf β
ββββββββββββββββββββββββββββββββββ
```
| offset | field | value |
|-----------|---------------|------------------------------------------------------|
| `[0..8)` | `kind` | `MessageKind` discriminant (see table below) |
| `[8..16)` | `payload_len` | byte length of the protobuf payload that follows |
The `kind` field tells the decoder how to interpret the payload:
| Value | `MessageKind` | Payload |
|------:|------------------------------|---------------------------------------------------------------------------------------|
| `0` | `End` | An `RrdFooter` protobuf (the optional footer β see below) |
| `1` | `SetStoreInfo` | A `SetStoreInfo` protobuf (announces a store) |
| `2` | `ArrowMsg` | An `ArrowMsg` protobuf wrapping a chunk's Arrow IPC bytes (optionally LZ4-compressed) |
| `3` | `BlueprintActivationCommand` | A `BlueprintActivationCommand` protobuf |
The outer payload bytes are always plain protobuf. Of the four kinds, only `ArrowMsg` can carry compressed data: its `compression` field tracks whether the wrapped Arrow IPC bytes are LZ4-compressed, so different `ArrowMsg`s in the same file can mix compressed and uncompressed Arrow IPC payloads.
The `compression` byte in the `StreamHeader`'s `EncodingOptions` is advisory only β per-message decoding does not consult it.
### Stream footer
The footer is written in two parts.
The first part is a regular framed message: an `End`-kind `MessageHeader` followed by the `RrdFooter` protobuf payload described in the [Footer](#footer) section.
It lives somewhere in the message stream β usually right before the file is closed β and is no different from any other framed message structurally.
The second part is the **`StreamFooter` trailer** at EOF. It is *not* a framed message: it is a raw structure whose job is to let readers jump straight to the `RrdFooter`(s) from the end of the file.
The trailer is a variable-length entry table β one 20-byte `StreamFooterEntry` per `RrdFooter` in the stream β followed by a fixed 12-byte tail that always sits at the very end of the file:
```
StreamFooter = num_entries Γ StreamFooterEntry + 12-byte static tail
ββββββββββββββββββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββββββββ
β entries[0..num_entries) β 20Β·num_entries B β static tail β 12 B β
ββββββββββββββββββββββββββββββββββββββββββββββββ΄ββββββββββββββββββββββββββ
EOF β 12 β 20Β·num_entries EOF β 12 EOF
```
Each `StreamFooterEntry` is 20 bytes:
| offset | field | value |
|------------|---------|--------------------------------------------------------------------------------|
| `[0..8)` | `start` | u64 LE β byte offset of the `RrdFooter` payload (after its own MessageHeader) |
| `[8..16)` | `len` | u64 LE β length of the `RrdFooter` payload |
| `[16..20)` | `crc32` | u32 LE β `xxh32(payload)` with the fixed seed `7850921` (`"RERUN"` in base-26) |
The static tail is the part with a known offset from EOF:
| offset (from EOF) | field | value |
|---------------------|---------------|------------------------------------|
| `[-12..-8)` | FourCC | `b"RRF2"` |
| `[-8..-4)` | identifier | `b"FOOT"` |
| `[-4..0)` | `num_entries` | u32 LE β number of entries above |
The CRC only covers the `RrdFooter` payload, not the surrounding `MessageHeader`, so it can be checked independently of message framing.
Reading the footer therefore boils down to:
```
1. seek EOF β 12 β read FourCC, identifier, num_entries
2. seek EOF β 12 β 20Β·num_entries β read num_entries entries
3. for each entry:
seek entry.start, read entry.len bytes
check xxh32(bytes, seed=7850921) == entry.crc32
decode protobuf RrdFooter
```
A file can legally have more than one trailer β that happens when streams are simply concatenated (`cat a.rrd b.rrd > both.rrd`), but tools like `rerun rrd merge` collapse them back into a single trailer with a single entry.
Files written without a footer (streaming sinks, legacy producers) skip the `End` message and the trailer entirely; readers detect their absence by the missing `FOOT` identifier and fall back to a linear scan.
## Stability
The format is split into two layers with different stability stories.
### Binary format
This concerns the binary structure of the RRD file.
The framing described in [File layout](#file-layout) is considered **stable** and we have no plans to change it.
Legacy RRDs whose `Version` field is older than `0.23` are currently rejected.
That's the cut-off below which we do not attempt to migrate at load time; a manual hop through an older Rerun SDK release is required.
The same is true for RRDs whose FourCC is `RRF0` or `RRF1`.
Should we need to break framing compatibility again, the FourCC will bump (`RRF3`, β¦) and load-time auto-migration will be provided.
The `rerun rrd migrate` CLI will also be available for offline batch conversion.
### Sorbet
We refer to the high-level data model specification as Sorbet.
Its reference implementation lives in the Rust `re_sorbet` crate.
This includes the chunk and footer schemas, as well as the high-level data model (timelines, archetypes, components, etc. β see [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md)).
Sorbet is versioned and **subject to change**, but `re_sorbet` performs in-memory migration to the current Sorbet version as chunks (and the footer manifest) are loaded.
Any CLI tool that rewrites an RRD (`rerun rrd merge`, `rerun rrd optimize`, `rerun rrd migrate`, β¦) emits chunks in the current Sorbet version, so a round-trip through any of these is also a migration.
Future changes to Sorbet will be auto-migrated in the same way.
# Chunks
A *Chunk* is the core datastructure at the heart of Rerun: it dictates how data gets logged, injected, stored, and queried.
A basic understanding of chunks is important in order to understand why and how Rerun and its APIs work the way they work.
## How Rerun stores data
All the data you send into Rerun is stored in chunks, always.
A chunk is an [Arrow](https://arrow.apache.org/)-encoded, column-oriented table of binary data:
A *Component Column* contains one or more [*Component Batches*](https://rerun.io/docs/concepts/logging-and-ingestion/batches.md), which in turn contain one or more instances (that is, a component is *always* an array). Each component batch corresponds to a single *Row ID* and one [time point per timeline](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md).
This design allows for keeping chunks within a target size range, even for recordings that combine low frequency but large data like point clouds or tensors (wide columns), with high frequency but small signals (tall columns).
Here's an excerpt from a real-world chunk (taken from the [Helix example](https://app.rerun.io/?url=https%3A%2F%2Fapp.rerun.io%2Fversion%2Flatest%2Fexamples%2Fdna.rrd)) (you might want to open [this image](https://static.rerun.io/a_real_chunk/2c4c16303dd1a04ba8ad8962ed85386a6568773e/full.png) in a new tab):
You can see that this matches very closely the diagram above:
* A single *control* column, that contains the unique row IDs.
* Multiple *time*/*index* columns (`log_tick`, `log_time`, `stable_time`).
* Multiple component columns (`Points3D:colors`, `Points3D:positions`, `Points3D:radii`).
Within each row of each component column, the individual cells are [*Component Batches*](https://rerun.io/docs/concepts/logging-and-ingestion/batches.md). Component batches are the atomic unit of data in Rerun.
The data in this specific chunk was logged with the following code:
```python
rr.set_time("stable_time", duration=time)
beads = [bounce_lerp(points1[n], points2[n], times[n]) for n in range(NUM_POINTS)]
colors = [[int(bounce_lerp(80, 230, times[n] * 2))] for n in range(NUM_POINTS)]
rr.log(
"helix/structure/scaffolding/beads",
rr.Points3D(beads, radii=0.06, colors=np.repeat(colors, 3, axis=-1))
)
```
You can learn more about chunks and how they came to be in [this blog post](https://rerun.io/blog/column-chunks#storage-is-based-around-chunks-of-component-columns).
## Getting chunks into Rerun
If you've used the Rerun SDK before, you know it doesn't actually force you to craft these chunks manually, which would be rather cumbersome!
How does one create and store chunks in Rerun, then?
### The row-oriented logging: `log`
The `log` API is generally [what we show in the getting-started guides](https://rerun.io/docs/getting-started/data-in#logging-our-first-points) since it's the easiest to use:
```python
"""
Update a scalar over time.
See also the `scalar_column_updates` example, which achieves the same
thing in a single operation.
"""
from __future__ import annotations
import math
import rerun as rr
rr.init("rerun_example_scalar_row_updates", spawn=True)
for step in range(64):
rr.set_time("step", sequence=step)
rr.log("scalars", rr.Scalars(math.sin(step / 10.0)))
```
The `log` API makes it possible to send data into Rerun on a row-by-row basis, without requiring any extra effort.
This row-oriented interface makes it very easy to integrate into existing codebase and just start logging data as it comes (hence the name).
Reference:
* [π Python `log`](https://ref.rerun.io/docs/python/stable/common/logging_functions/#rerun.log)
* [π¦ Rust `log`](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.log)
* [π C++ `log`](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a7badac918d44d66e04e948f38818ff11)
But if you're handing a bunch of rows of data over to Rerun, how does it end up neatly packaged in columnar chunks?
#### How are these rows turned into columns?
Before logging data, you can use the `rr.set_time_` APIs to update the SDK's time context with timestamps for custom timelines.
For example, `rr.set_time("frame", sequence=42)` will set the "frame" timeline's current value to 42 in the time context.
When you later call `rr.log`, the SDK will generate a row id and a value for the built-in `log_time` timeline (enabled by default), as well as `log_tick` if you have opted in to it.
It will also grab the current values for any custom timelines from the time context.
Any data passed to `rr.log` becomes component batches.
The row id, timestamps, and logged component batches are then encoded as Apache Arrow arrays and together make up a row.
That row is then passed to a batcher, which appends the values from the row to the current chunk for the entity path.
The current chunk is then sent to its destination, either periodically or as soon as it crosses a size threshold.
Building up small column chunks before sending from the SDK trades off a small amount of latency and memory use in favor of more efficient transfer and ingestion.
You can read about how to configure the batcher [here](https://rerun.io/docs/reference/sdk/micro-batching.md).
### The column-oriented logging: `send_columns`
The `log` API showcased above is designed to extract data from your running code as it's being generated. It is, by nature, *row-oriented*.
If you already have data stored in something more *column-oriented*, it can be both a lot easier and more efficient to send it to Rerun in that form directly.
This is what the `send_columns` API is for: it lets you efficiently update the state of an entity over time, sending data for multiple index and component columns in a single operation.
> [!WARNING]
> `send_columns` API bypasses the time context and [micro-batcher](https://rerun.io/docs/reference/sdk/micro-batching.md).
>
> In contrast to the `log` API, `send_columns` does NOT add any other timelines to the data. Neither the built-in timelines `log_time` and `log_tick`, nor any [user timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md). Only the timelines explicitly included in the call to `send_columns` will be included.
```python
"""
Update a scalar over time, in a single operation.
This is semantically equivalent to the `scalar_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_scalar_column_updates", spawn=True)
times = np.arange(0, 64)
scalars = np.sin(times / 10.0)
rr.send_columns(
"scalars",
indexes=[rr.TimeColumn("step", sequence=times)],
columns=rr.Scalars.columns(scalars=scalars),
)
```
Reference:
* [π Python `send_columns`](https://ref.rerun.io/docs/python/stable/common/columnar_api/#rerun.send_columns)
* [π¦ Rust `send_columns`](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.send_columns)
* [π C++ `send_columns`](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a7e326526d1473c02fcb2ed94afe6da69)
### Sending actual chunks: `send_chunks`
The `Chunk` data structure described above is also exposed as a Python class.
You can build a chunk from, e.g., time/component columns, inspect or transform existing chunks, and forward chunks to a recording stream with `send_chunks`:
```python
"""Build a `Chunk` with `Chunk.from_columns` and send it via `send_chunks`."""
from __future__ import annotations
import rerun as rr
import rerun.experimental as rrx
rr.init("rerun_example_build_chunk")
chunk = rrx.Chunk.from_columns(
"/points",
indexes=[rr.TimeColumn("frame", sequence=[0, 1, 2])],
columns=rr.Points3D.columns(
positions=[[1, 0, 0], [0, 1, 0], [0, 0, 1]],
radii=[0.1, 0.2, 0.3],
),
)
# Chunks can be inspected in many ways, including a text representation of
# its content
print(chunk)
rrx.send_chunks(chunk)
```
Alternatively, chunks can be created from an existing Arrow [`RecordBatch`](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html) using [`Chunk.from_record_batch`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk.from_record_batch):
```python
# Create an index column.
frame = pa.array([0, 1, 2], type=pa.int64())
# Create two component columns.
positions_datatype = pa.list_(
pa.list_(pa.field("item", pa.float32(), nullable=False), 3)
)
left = pa.array(
[[[1.0, 0.0, 0.0]], [[2.0, 0.0, 0.0]], [[3.0, 0.0, 0.0]]],
type=positions_datatype,
)
right = pa.array(
[[[0.0, 1.0, 0.0]], [[0.0, 2.0, 0.0]], [[0.0, 3.0, 0.0]]],
type=positions_datatype,
)
# The `/entity:Archetype:component` column-name convention tells
# `from_record_batch` which entity and component each column maps to.
batch = pa.RecordBatch.from_arrays(
[frame, left, right],
names=["frame", "/left:Points3D:positions", "/right:Points3D:positions"],
)
chunks = rrx.Chunk.from_record_batch(batch, index="frame")
for chunk in chunks:
print(chunk)
```
`send_chunks` also accepts iterables of chunks, as well as instances of [`LazyChunkStream`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyChunkStream), [`ChunkStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.ChunkStore), and [`LazyStore`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.LazyStore).
For example, to forward every chunk of an existing RRD into a new recording stream:
```python
"""Send chunks loaded from an RRD into a recording stream."""
import sys
import rerun as rr
import rerun.experimental as rrx
path_to_rrd = sys.argv[1]
# NOTE: This is specifically demonstrating how to forward chunks from an RRD
# into the viewer.
# If you just want to view an RRD file, use the simpler `rr.log_file()`
# function instead:
# rr.log_file("path/to/file.rrd", spawn=True)
reader = rrx.RrdReader(path_to_rrd)
entry = reader.recordings()[0]
rr.init(entry.application_id, recording_id=entry.recording_id, spawn=True)
rrx.send_chunks(reader.store())
```
Like `send_columns`, this path bypasses the time context and the [micro-batcher](https://rerun.io/docs/reference/sdk/micro-batching.md): chunks are forwarded as-is, with whatever timelines they were built with.
See the [Chunk Processing API](https://rerun.io/docs/concepts/logging-and-ingestion/chunk-processing-api.md) for building ingestion, transformation, and conversion pipelines out of these primitives.
Reference:
* [π Python `Chunk`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk)
* [π Python `Chunk.from_columns`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk.from_columns)
* [π Python `Chunk.from_record_batch`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk.from_record_batch)
* [π Python `send_chunks`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.send_chunks)
### Dataframe logging: `Chunk.from_dataframe` and `send_dataframe`
[`rr.send_dataframe`](https://ref.rerun.io/docs/python/stable/common/columnar_api/#rerun.send_dataframe) and the related [`Chunk.from_dataframe`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk.from_dataframe) extend the single record batch equivalent and accept a full PyArrow [`Table`](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html) (or a [`RecordBatchReader`](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatchReader.html), or any Arrow-C-stream object such as a `datafusion.DataFrame`) carrying any number of entities, and yields the chunks of each record batch in turn.
To map columns of the dataframe to Rerun timelines and components, the dataframe must carry the same `rerun:*` metadata as above.
For example, here we hand-craft a dataframe containing a Points3D entity:
```python
# An index columnβ¦
index = pa.array([0, 1, 2], type=pa.int64())
# β¦and a component column. Each row is a list (one component batch per row).
positions = pa.array(
[
[[1.0, 0.0, 0.0]],
[[0.0, 1.0, 0.0]],
[[0.0, 0.0, 1.0]],
],
type=pa.list_(pa.list_(pa.field("item", pa.float32(), nullable=False), 3)),
)
# Tag each column with the `rerun:*` metadata keys that `Chunk.from_dataframe`
# recognizes.
schema = pa.schema([
pa.field(
"frame",
index.type,
metadata={b"rerun:index_name": b"frame", b"rerun:kind": b"index"},
),
pa.field(
"/points:Points3D:positions",
positions.type,
metadata={
b"rerun:entity_path": b"/points",
b"rerun:archetype": b"rerun.archetypes.Points3D",
b"rerun:component": b"Points3D:positions",
b"rerun:component_type": b"rerun.components.Position3D",
b"rerun:kind": b"data",
},
),
])
table = pa.Table.from_arrays([index, positions], schema=schema)
```
`Chunk.from_dataframe` then interprets that metadata and yields one chunk per entity path:
```python
chunks = list(rrx.Chunk.from_dataframe(table))
for chunk in chunks:
print(chunk)
```
`rr.send_dataframe` is a thin logging convenience wrapper over `Chunk.from_dataframe`: it builds those same chunks and forwards them to the active recording stream in one call.
```python
rr.send_dataframe(table)
```
Like `send_columns`, it bypasses the time context and the [micro-batcher](https://rerun.io/docs/reference/sdk/micro-batching.md) β only timelines explicitly present in the table are added.
Manually crafting the required metadata is obviously inconvenient.
This API is instead designed to compose with [dataframe queries](https://rerun.io/docs/concepts/query-and-transform/dataframe-queries.md), which produce dataframes already populated with metadata derived from the originally queried data.
Reference:
* [π Python `Chunk.from_dataframe`](https://ref.rerun.io/docs/python/stable/experimental/#rerun.experimental.Chunk.from_dataframe)
* [π Python `send_dataframe`](https://ref.rerun.io/docs/python/stable/common/columnar_api/#rerun.send_dataframe)
* [π Python `send_record_batch`](https://ref.rerun.io/docs/python/stable/common/columnar_api/#rerun.send_record_batch)
# Video
A stream of images (like those produced by a camera) can be logged to Rerun in several different ways:
* Uncompressed, as many [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)s
* Compressed as many [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)s, using e.g. JPEG.
* Compressed as a single [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md), using e.g. MP4.
* Compressed as a series of encoded video samples using [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md), using e.g. H.264 encoded frames.
These alternatives range on a scale of "simple, lossless, and big" to "complex, lossy, and small".
If you want lossless encoded images (with no compression artifacts), then you should log each video frame as `Image`.
This will use up a lot of space and bandwidth. You can also encode them as PNG and log them as `EncodedImage`,
though it should be noted that PNG encoding usually does very little for the file size of photographic images.
If you want to reduce bandwidth and storage cost, you can encode each frame as a JPEG and log it using `EncodedImage`. This can easily reduce the file sizes by almost two orders of magnitude with minimal perceptual loss.
This is also very simple to do, and the Python logging SDK has built-in support for it using [`Image.compress`](https://ref.rerun.io/docs/python/0.18.2/common/archetypes/#rerun.archetypes.Image.compress).
Finally, for the best compression ratio, you can encode the images as an encoded video.
There are two options to choose from:
* Raw video frames [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
* Video files using [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md)
> [!WARNING]
> Do not use compressed video if you need accurate pixel replication:
> this is not only due to the obvious detail loss on encoding,
> but also since the exact _display_ of the same video is not consistent across platforms and decoder versions.
## Streaming video / raw encoded video frames
The following example illustrates how to encode uncompressed video frames (represented by `numpy` arrays)
using [`pyAV`](https://github.com/PyAV-Org/PyAV) into H.264 and directly log them to Rerun using [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md).
```python
"""Video encode images using av and stream them to Rerun."""
import av
import numpy as np
import numpy.typing as npt
import rerun as rr
fps = 30
duration_seconds = 4
width = 480
height = 320
ball_radius = 30
codec = rr.VideoCodec.H265 # rr.VideoCodec.H264
formats = {rr.VideoCodec.H265: "hevc", rr.VideoCodec.H264: "h264"}
encoders = {rr.VideoCodec.H265: "libx265", rr.VideoCodec.H264: "libx264"}
def create_example_video_frame(frame_i: int) -> npt.NDArray[np.uint8]:
img = np.zeros((height, width, 3), dtype=np.uint8)
for h in range(height):
img[h, :] = [
0,
int(100 * h / height),
int(200 * h / height),
] # Blue to purple gradient.
x_pos = width // 2 # Center horizontally.
y_pos = height // 2 + 80 * np.sin(2 * np.pi * frame_i / fps)
y, x = np.ogrid[:height, :width]
r_sq = (x - x_pos) ** 2 + (y - y_pos) ** 2
img[r_sq < ball_radius**2] = [255, 200, 0] # Gold color
return img
rr.init("rerun_example_video_stream_synthetic")
# Setup encoding pipeline.
av.logging.set_level(av.logging.VERBOSE)
container = av.open(
"/dev/null", "w", format=formats[codec]
) # Use AnnexB H.265 stream.
stream = container.add_stream(encoders[codec], rate=fps)
# Type narrowing
assert isinstance(stream, av.video.stream.VideoStream)
stream.width = width
stream.height = height
# TODO(#10090): Rerun Video Streams don't support b-frames yet.
# Note that b-frames are generally not recommended for low-latency streaming
# and may make logging more complex.
stream.max_b_frames = 0
# Log codec only once as static data (it naturally never changes).
# This isn't strictly necessary, but good practice.
rr.log("video_stream", rr.VideoStream(codec=codec), static=True)
# Generate frames and stream them directly to Rerun.
for frame_i in range(fps * duration_seconds):
img = create_example_video_frame(frame_i)
frame = av.VideoFrame.from_ndarray(img, format="rgb24")
for packet in stream.encode(frame):
if packet.pts is None:
continue
rr.set_time("time", duration=float(packet.pts * packet.time_base))
rr.log("video_stream", rr.VideoStream.from_fields(sample=bytes(packet)))
# Flush stream.
for packet in stream.encode():
if packet.pts is None:
continue
rr.set_time("time", duration=float(packet.pts * packet.time_base))
rr.log("video_stream", rr.VideoStream.from_fields(sample=bytes(packet)))
```
Using [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) requires deeper knowledge of the encoding process
but unlike [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md),
allows the Rerun Viewer to show incomplete or open ended video streams.
In contrast, [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md) requires the entire
video asset file to be in Viewer memory before decoding can begin.
Refer to the [video camera streaming](https://github.com/rerun-io/rerun/blob/latest/examples/python/camera_video_stream) example to learn how to stream live video to Rerun.
For more details on how to query and decode video streams from Rerun, see our [query video streams how-to](https://rerun.io/docs/howto/query-and-transform/query_videos.md).
Current limitations of `VideoStream`:
* [#9815](https://github.com/rerun-io/rerun/issues/9815): Decoding on native is generally slower than decoding in the browser right now.
This can cause increased latency and in some cases may even stop video playback.
* [#10090](https://github.com/rerun-io/rerun/issues/10090): B-frames are not yet supported for [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md).
* [#10422](https://github.com/rerun-io/rerun/issues/10422): [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md) does not yet work with [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md).
### Export MP4 from RRD (remuxing)
Sample data from [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) can be queried
and remuxed to mp4 without re-encoding the video as demonstrated in [this sample](https://github.com/rerun-io/rerun/blob/latest/docs/snippets/all/archetypes/video_stream_query_and_mux.py).
Check the [doc page on retrieving data](https://rerun.io/docs/howto/query-and-transform/get-data-out.md) to learn more about dataframe queries in general.
## Video files
You can use [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md) to log readily encoded video files.
Rerun ignores the timestamp at which the video asset itself is logged and requires you
to log [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md) to establish a
correlation of video time to the Rerun timeline.
To ease this, the SDK's `read_frame_timestamps_nanos` utility allows to read out timestamps from in-memory video assets:
```python
"""Log a video asset using automatically determined frame references."""
import sys
import rerun as rr
if len(sys.argv) < 2:
# TODO(#7354): Only mp4 is supported for now.
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset_video_auto_frames", spawn=True)
# Log video asset which is referred to by frame references.
video_asset = rr.AssetVideo(path=sys.argv[1])
rr.log("video", video_asset, static=True)
# Send automatically determined video frame timestamps.
frame_timestamps_ns = video_asset.read_frame_timestamps_nanos()
rr.send_columns(
"video",
# Note timeline values don't have to be the same as the video timestamps.
indexes=[rr.TimeColumn("video_time", duration=1e-9 * frame_timestamps_ns)],
columns=rr.VideoFrameReference.columns_nanos(frame_timestamps_ns),
)
```
[#7354](https://github.com/rerun-io/rerun/issues/7354): Currently, only MP4 files are supported.
## Codec support in detail
### Overview
Codec support varies in the web & native viewer:
| | Browser | Native |
| ---------- | ------- | ------ |
| AV1 | β
| π§ |
| H.264/avc | β
| β
|
| H.265/hevc | π§ | β
|
| VP8 | β
| β
|
| VP9 | β
| β
|
Details see below.
When choosing a codec, we recommend [AV1](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#av1),
as it seems to have the best overall playback support while also having very high compression quality.
Since AV1 can have very long encoding times, it is often not suitable for streaming.
In cases where encoding time matters, we recommend H.264/avc.
### Native viewer
#### AV1
AV1 is supported out of the box using a software decoder paired with gpu based image conversion.
Current limitations:
* [#7755](https://github.com/rerun-io/rerun/issues/7755): AV1 is supported on all native builds exception on Linux ARM.
#### H.264/avc, H.265/hevc, VP8 & VP9
H.264/avc, H.265/hevc, VP8, and VP9 are supported via a separately installed `FFmpeg` binary, requiring a minimum version of `5.1`.
The viewer does intentionally not come bundled with `FFmpeg` to avoid licensing issues.
By default rerun will look for a system installed `FFmpeg` installation in `PATH`,
but you can specify a custom path in the viewer's settings.
If you select a video that failed to play due to missing or incompatible `FFmpeg` binaries it will offer a download link to a build of `FFmpeg` for your platform.
### Web viewer
Video playback in the Rerun Web Viewer is done using the browser's own video decoder, so the exact supported codecs depend on your browser.
Overall, we recommend using Chrome or another Chromium-based browser, as it seems to have the best video support as of writing.
For decoding video in the Web Viewer, we use the [WebCodecs API](https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API).
This API enables us to take advantage of the browser's hardware accelerated video decoding capabilities.
It is implemented by all modern browsers, but with varying levels of support for different codecs, and varying levels of quality.
When it comes to codecs, we aim to support any codec which the browser supports, but
we currently cannot guarantee that all of them will work. For more information about
which codecs are supported by which browser, see [Video codecs on MDN](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#codec_details).
We tested the following codecs in more detail:
| | Linux Firefox | Linux Chrome[^1] | macOS Firefox | macOS Chrome | macOS Safari | Windows Firefox | Windows Chrome[^2] |
| ---------- | ------------- | ---------------- | ------------- | ------------ | ------------ | --------------- | ------------------ |
| AV1 | β
| β
| β
| β
| π§[^3] | β
| β
|
| H.264/avc | β
| β
| β
| β
| β
| β
| β
|
| H.265/hevc | β | β | β | β
| π§[^4] | β | π§[^5] |
| VP8 | β
| β
| β
| β
| β | | |
| VP9 | β
| β
| β
| β
| β | | |
[^1]: Any Chromium-based browser should work, but we don't test all of them.
[^2]: Chrome on Windows has been observed to stutter on playback. It can be mitigated by [using software decoding](https://rerun.io/docs/getting-started/install-rerun/troubleshooting.md), but this may lead to high memory usage. See [#7595](https://github.com/rerun-io/rerun/issues/7595).
[^3]: Safari/WebKit does not support AV1 decoding except on [Apple Silicon devices with hardware support](https://webkit.org/blog/14445/webkit-features-in-safari-17-0/).
[^4]: Safari/WebKit has been observed stuttering when playing `hvc1` but working fine with `hevc1`. Despite support being advertised Safari 16.5 has been observed not support H.265 decoding.
[^5]: Only supported if hardware encoding is available. Therefore always affected by Windows stuttering issues, see above.
Beyond this, for best compatibility we recommend:
* prefer YUV over RGB & monochrome formats
* don't use more than 8bit per color channel
* keep resolutions at 8k & lower (see also [#3782](https://github.com/rerun-io/rerun/issues/3782))
## Other limitations
There are still some limitations to encoded Video in Rerun which will be addressed in the future:
* [#7594](https://github.com/rerun-io/rerun/issues/7594): HDR video is not supported
* [#5181](https://github.com/rerun-io/rerun/issues/5181): There is no audio support
* There is no video encoder in the Rerun SDK, so you need to create the video stream or file yourself.
Refer to the [video camera streaming](https://github.com/rerun-io/rerun/blob/latest/examples/python/camera_video_stream) example to learn how to encode video using [`pyAV`](https://github.com/PyAV-Org/PyAV).
## Links
* [Web video codec guide, by Mozilla](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs)
# MCAP Decoders Explained
MCAP processing in Rerun uses a decoder architecture where each decoder represents a different way to interpret and extract data from the same MCAP source.
By default, when opening a file Rerun analyzes an MCAP file to determine which decoders are active to provide the most comprehensive view of your data, while avoiding duplication.
You can specify which decoders to use during conversion, allowing you to extract exactly the information you need for your analysis.
## Understanding decoders with an example
When multiple decoders are enabled, they each process the same messages independently, creating different component types on identical entity paths. This can result in data duplication β for instance, enabling both `raw` and `protobuf` decoders stores the same message as both structured field data and raw binary blobs.
Consider an MCAP file from a ROS2 robot containing sensor data on the topic `/robot/camera/image_raw` with ROS2 `sensor_msgs/msg/Image` messages:
- With only the `ros2msg` decoder: Creates an [Image](https://rerun.io/docs/reference/types/archetypes/image.md) archetype for direct visualization in Rerun's viewer
- With only the `raw` decoder: Creates an [McapMessage](https://rerun.io/docs/reference/types/archetypes/mcap_message.md) containing the original CDR-encoded message bytes
- With both decoders enabled: All representations coexist on the same entity path `/robot/camera/image_raw`
## Schema and statistics decoders
The `schema` decoder extracts structural information about the MCAP file's organization, creating metadata entities that describe channel definitions, topic names with their message types, and schema definitions. This decoder is particularly useful for understanding unfamiliar MCAP files or getting an overview of available topics and channels before deeper processing.
The `stats` decoder computes file-level metrics and statistics, creating entities with message counts per channel, temporal ranges, file size information, and data rate analysis. This gives you insight into the scale and characteristics of your dataset for quality assessment and planning storage requirements.
## Message interpretation decoders
### Semantic interpretation
The `ros2msg` and `foxglove` decoders provide semantic interpretation and visualization of standard ROS 2 and Foxglove message types, creating meaningful Rerun visualization archetypes from data. Unlike the `protobuf` decoder, this decoder understands the semantics of the messages and creates appropriate visualizations: images become [Image](https://rerun.io/docs/reference/types/archetypes/image.md), point clouds become [Points3D](https://rerun.io/docs/reference/types/archetypes/points3d.md), IMU messages become [SeriesLines](https://rerun.io/docs/reference/types/archetypes/series_lines.md) with the data plotted over time, and so on.
See [Message Formats](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md) for the complete list of supported message types.
### Protobuf decoding
The `protobuf` decoder automatically decodes protobuf-encoded messages using reflection, creating structured component data based on the protobuf schema. Message fields become Rerun components that you can query and analyze.
However, this decoder provides structured access without semantic visualization meaning. While the data becomes queryable, it won't automatically appear as meaningful visualizations like images or point clouds, it gives you the data structure, not the visual interpretation.
## The raw decoder
The `raw` decoder preserves the original message bytes without any interpretation, creating blob entities containing the unprocessed message data. Each message appears as a binary blob that can be accessed programmatically for custom analysis tools.
## Recording info
The `recording_info` decoder extracts metadata about the recording session and capture context, creating metadata entities with information about recording timestamps, source system details, and capture software versions.
## The URDF option
The `urdf` option uses Rerun's built-in URDF loader if there is a ROS 2 string topic named `/robot_description`, logging the robot model as static 3D geometry. In this MCAP workflow, joint transforms are not loaded from the URDF itself; they are expected to come from TF topics in the MCAP (e.g. `/tf` or `/tf_static`).
For general information about how to load URDF files, see [here](https://rerun.io/docs/howto/logging-and-ingestion/urdf.md).
## Decoder selection and performance
### Selecting decoders
By default, Rerun processes MCAP files with all decoders active. You can control which decoders are used when [converting MCAP files via the CLI](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/cli-reference.md) using the `-d` flag:
```bash
# Use only specific decoders
rerun mcap convert input.mcap -d protobuf -d stats -o output.rrd
# Use multiple decoders for different perspectives
rerun mcap convert input.mcap -d ros2msg -d raw -d recording_info -o output.rrd
# Add robot geometry from ROS robot_description topics
rerun mcap convert input.mcap -d ros2msg -d urdf -o output.rrd
```
## Accessing decoder data
Each decoder creates different types of components on entity paths (derived from MCAP channel topics) that can be accessed through Rerun's SDK:
- Data from the `ros2msg` decoder and supported Foxglove messages appears as native Rerun visualization archetypes (see [here](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md) for an overview)
- Other data from the `protobuf` or `ros2_reflection` decoders appears as structured components that can be queried by field name or manually added to certain views ([example](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/message-formats.md))
- Data from the `raw` decoder appears as blob components containing the original message bytes
- Data from the `urdf` option appears as static 3D robot geometry loaded from the ROS 2 `/robot_description` topic
- Metadata from `schema`, `stats`, and `recording_info` decoders appears as dedicated metadata entities
For more information on querying data and working with archetypes, see the [Data Queries documentation](https://rerun.io/docs/howto/query-and-transform/get-data-out.md).
Each of these decoders contributes their own [chunks](https://rerun.io/docs/concepts/logging-and-ingestion/chunks.md) to the Rerun-native data.
Below is a table showing the mapping between MCAP data and Rerun components:
| MCAP Data | Rerun component | Description |
| ---------------- | ------------------------------- | ----------------------------------------------------------------------------- |
| Schema name | `mcap.Schema:name` | Message type name from schema definition |
| Schema data | `mcap.Schema:data` | Raw schema definition (protobuf, ROS2 msg, etc.) |
| Schema encoding | `mcap.Schema:encoding` | Schema format type |
| | | |
| Channel topic | `mcap.Channel:topic` | Topic name from MCAP channel |
| Channel ID | `mcap.Channel:id` | Numeric channel identifier |
| Message encoding | `mcap.Channel:message_encoding` | Encoding format (e.g., `protobuf`, `cdr`) |
| | | |
| Statistics | `mcap.Statistics` | File-level metrics like message counts and time ranges |
| Raw message data | `mcap.Message:data` | Unprocessed message bytes stored as binary blobs, handled by the `raw` decoder. |
# Supported Message Formats
Rerun provides automatic visualization for common message types in MCAP files:
* ROS 2 messages
* Foxglove schemas (Protobuf)
## Overview
This table shows an overview of the ROS 2 and Foxglove message schemas that are automatically converted to Rerun archetypes.
We are continually adding support for more standard message types.
| Modality | ROS 2 | Foxglove Protobuf | Rerun Archetypes |
| --- | --- | --- | --- |
| Raw image | `sensor_msgs/Image` | `RawImage` | [Image](https://rerun.io/docs/reference/types/archetypes/image.md), [DepthImage](https://rerun.io/docs/reference/types/archetypes/depth_image.md) |
| Encoded image | `sensor_msgs/CompressedImage` | `CompressedImage` | [EncodedImage](https://rerun.io/docs/reference/types/archetypes/encoded_image.md), [EncodedDepthImage](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md) |
| Video | `sensor_msgs/CompressedImage` (h264) | `CompressedVideo` | [VideoStream](https://rerun.io/docs/reference/types/archetypes/video_stream.md) |
| Camera calibration | `sensor_msgs/CameraInfo` | `CameraCalibration` | [Pinhole](https://rerun.io/docs/reference/types/archetypes/pinhole.md) |
| Point cloud | `sensor_msgs/PointCloud2` | `PointCloud` | [Points3D](https://rerun.io/docs/reference/types/archetypes/points3d.md) |
| Geo points | `sensor_msgs/NavSatFix` | `LocationFix`, `LocationFixes` | [GeoPoints](https://rerun.io/docs/reference/types/archetypes/geo_points.md) |
| Transforms | `tf2_msgs/TFMessage` | `FrameTransform`, `FrameTransforms` | [Transform3D](https://rerun.io/docs/reference/types/archetypes/transform3d.md) |
| Poses | `geometry_msgs/PoseStamped` | `PoseInFrame`, `PosesInFrame` | [InstancePoses3D](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md) |
| Coordinate frame | `.frame_id` field in `std_msgs/Header` | `.frame_id` field | [CoordinateFrame](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)
| Magnetic field | `sensor_msgs/MagneticField` | - | [Arrows3D](https://rerun.io/docs/reference/types/archetypes/arrows3d.md) |
| Misc. scalar sensor data | `sensor_msgs/Imu`, `sensor_msgs/JointState`, `sensor_msgs/Temperature`, `sensor_msgs/FluidPressure`, `sensor_msgs/RelativeHumidity`, `sensor_msgs/Illuminance`, `sensor_msgs/Range`, `sensor_msgs/BatteryState`, `sensor_msgs/Joy` | - *(usually covered via custom schemas, see [Schema reflection](#schema-reflection) below on this page)* | [Scalars](https://rerun.io/docs/reference/types/archetypes/scalars.md) |
| Text | `std_msgs/String` | - | [TextDocument](https://rerun.io/docs/reference/types/archetypes/text_document.md) |
| Log messages | `rcl_interfaces/Log` | `Log` | [TextLog](https://rerun.io/docs/reference/types/archetypes/text_log.md) |
| 2D grid map |Β `nav_msgs/OccupancyGrid` | - | [GridMap](https://rerun.io/docs/reference/types/archetypes/grid_map.md) |
| 3D voxel grid map | `nav2_msgs/VoxelGrid` | `VoxelGrid` | [VoxelGridMap](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md) |
### Timelines
The MCAP importer adds [timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md) based on the message timestamps.
In addition to the `message_log_time` and `message_publish_time` timestamps that are part of every MCAP message, we also add timelines with the application-specific timestamps from ROS and Foxglove schemas.
#### ROS
Most ROS message payloads have an additional [`Header`](https://rerun.io/en/noetic/api/std_msgs/html/msg/Header.html) that may also contain timestamp information. These timestamps are put onto specific `ros2_*` timelines.
Timestamps within Unix time range (1990-2100) create a `ros2_timestamp` timeline. Values outside this range create a `ros2_duration` timeline representing relative time from custom epochs.
#### Foxglove
Data from schemas containing a `.timestamp` field is put onto a `timestamp` timeline.
### Transforms (TF)
Transform messages are converted to [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md), with `parent_frame` and `child_frame` set according to the `frame_id` and `child_frame_id` of each `geometry_msgs/TransformStamped` contained in the message's `transforms` list.
The timestamps of the individual transforms are put onto the respective timelines, allowing the viewer to resolve the spatial relationships between frames over time similar to a TF buffer in ROS.
> You can read more about how Rerun handles transforms and "TF-style" frame names [here](https://rerun.io/docs/concepts/transforms#named-transform-frames).
To see the transforms in the viewer, you can select the entity corresponding to the topic and add a visualizer for `TransformAxes3D` as shown in the video here.
If you have transforms that correspond to joints in a robot model, you can also read more about how to load `URDF` models into a recording [here](https://rerun.io/docs/howto/logging-and-ingestion/urdf#load-urdf-into-an-existing-recording).
### Poses and frame IDs
Pose messages are converted to [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md) with a [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md) on the same entity path.
Just like `Transform3D`, you can visualize these poses in the viewer by selecting the entity and adding a `TransformAxes3D` visualizer in the selection panel.
Note that the visualization requires the parent coordinate frame of the pose to be known, i.e. part of the transform hierarchy of your data.
[`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)s are also used for other message types that are supported by the `ros2msg` layer, if they have an [`std_msgs/Header`](https://docs.ros2.org/foxy/api/std_msgs/msg/Header.html) with a `frame_id`.
For data that can be visualized in 3D views (e.g. point clouds), this means that the viewer takes the respective coordinate frame's transform into account and renders the data relative to it.
## Schema reflection
MCAP files allow for arbitrary custom message payloads, so you might have other message types in your files than the set of ROS 2 or Foxglove messages that Rerun automatically converts to archetypes.
Rerun's MCAP importer automatically decodes unknown Protobuf or ROS 2 messages using schema reflection.
This means that you won't get Rerun archetypes out of the box, but the message fields become queryable components (e.g. for training data curation via the Rerun SDK, see [here](https://rerun.io/docs/concepts/logging-and-ingestion/mcap/decoders-explained.md)).
Depending on the contents of your data, you can still manually add visualizers for certain fields to your blueprint, e.g. a time-series view for scalars or a dataframe view.
You can also use [Lenses](https://rerun.io/docs/concepts/query-and-transform/lenses.md) to attach Rerun semantics to the reflected data.
### Example: time-series plot for custom message scalars
Here's an example with a custom Protobuf message that contains a robot gripper state.
After adding a new timeseries view via the blueprint panel, we select the Protobuf fields from our MCAP that we want to view through the view's selection panel:
### View decoded message
Each entity that was decoded from an unknown MCAP message via reflection has an `.message` component, which contains queryable struct fields.
You can see this also in the selection panel:
## ROS1 message types
ROS 1 data is not supported for semantic interpretation through any decoder.
The `raw` and `schema` decoders are able to preserve the original bytes and structure of ROS 1 messages in MCAP files, but Rerun will not convert them to visualization archetypes.
We don't plan to add support for ROS 1 in Rerun, as it has reached [end-of-life](https://www.ros.org/blog/noetic-eol/) in May 2025.
But if you have legacy ROS 1 data and want to migrate it to modern formats, we recommend to try external tools like [`rosbags`](https://ternaris.gitlab.io/rosbags/).
For example, this command converts a ROS 1 `.bag` to a ROS 2 CDR-encoded `.mcap` that Rerun can import like any other supported ROS 2 recording:
```bash
rosbags-convert --src my_data_ros1.bag --dst my_data_ros2 --dst-storage mcap
rerun my_data_ros2/my_data_ros2.mcap
```
Please refer to the `rosbags` documentation for further information.
## Adding support for new types
To request support for additional message types:
- [File a GitHub issue](https://github.com/rerun-io/rerun/issues) requesting the specific message type
- Join the Rerun community on [Discord](https://discord.gg/PXtCgFBSmH) to discuss and provide feedback on message support priorities. Or if you're open for a conversation, [sign up here](https://rerun.io/feedback)
# CLI Reference for MCAP
This reference guide covers all command-line options and workflows for working with MCAP files in Rerun.
## Basic commands
### Direct viewing
Open MCAP files directly in the Rerun Viewer:
```bash
# View a single MCAP file
rerun data.mcap
# View multiple specific files
rerun file1.mcap file2.mcap file3.mcap
# Use glob patterns to load all MCAP files in a directory
rerun recordings/*.mcap
# Recursively load all MCAP files from a directory
rerun mcap_data/
```
### File conversion
Convert MCAP files to Rerun's native RRD format:
```bash
# Convert MCAP to RRD format for faster loading
rerun mcap convert input.mcap -o output.rrd
# Convert with custom output location
rerun mcap convert data.mcap -o /path/to/output.rrd
```
## Decoder selection
### Using specific decoders
Control which processing decoders are applied during conversion:
```bash
# Use only protobuf decoding and file statistics
rerun mcap convert input.mcap -d protobuf -d stats -o output.rrd
# Use only ROS2 semantic interpretation for robotics data
rerun mcap convert input.mcap -d ros2msg -o output.rrd
# Add robot geometry from ROS robot_description topics
rerun mcap convert input.mcap -d ros2msg -d urdf -o output.rrd
# Combine multiple decoders for comprehensive data access
rerun mcap convert input.mcap -d ros2msg -d raw -d recording_info -o output.rrd
```
### Available decoder options
Decoding:
- **`raw`**: Preserve original message bytes
- **`schema`**: Extract metadata and schema information
- **`stats`**: Compute file and channel statistics into RRD `__mcap_properties`
- **`metadata`**: Extract metadata records into RRD `__mcap_metadata`, if present
- **`attachments`**: Extract MCAP attachment records into static data under `__mcap_attachments`
- **`protobuf`**: Decode protobuf messages using into generic Arrow data without Rerun visualization components
- **`recording_info`**: Extract recording session metadata into RRD `__mcap_properties`
- **`urdf`**: Use Rerun's built-in URDF loader when a ROS 2 `/robot_description` topic is present
Semantic:
- **`foxglove`**: Semantic interpretation of Foxglove Protobuf messages
- **`ros2msg`**: Semantic interpretation of ROS2 messages
### Default behavior
When no `-d` flags are specified, all available decoders are used:
```bash
# These commands are equivalent (default uses all decoders):
rerun mcap convert input.mcap -o output.rrd
rerun mcap convert input.mcap \
-d raw \
-d attachments \
-d schema \
-d stats \
-d metadata \
-d protobuf \
-d recording_info \
-d urdf \
-d ros2msg \
-d foxglove \
-o output.rrd
```
# Roadmap
Rerun is building a data management and visualization engine for multimodal data that changes over time.
We aim to make it fast, simple to use, and easy to adapt and integrate into your existing workflows.
Open an issue or pull request on [GitHub](https://github.com/rerun-io/rerun) or join us on [Discord](https://discord.gg/PXtCgFBSmH) to let the community know what you'd like to see. Or if you're open for a conversation, [sign up here](https://rerun.io/feedback).
This page is meant to give an high level overview of ongoing and planned work. This roadmap is subject to change; GitHub will be the most authoritative source for active development.
## We continually work on
- Performance improvements
- UX & DX improvements
- Supporting more data types
- Rerun Hub features (commercial)
- Get in touch on hi@rerun.io if you're interested in becoming a design partner
## Roadmap of major feature areas
### Near term
- Improving our data ingestion and interpretation flexibility, especially through initial support for common ROS2 messages in MCAP files
- Greater capabilities around sharing links to data
- h.265 video streaming support
- An in-memory catalog to make recording file management simpler
### Medium term
- Filtering in table and dataframe views
- Configurable data interpretability (e.g. MCAP files with custom messages)
- Including _data blueprints_ that define and store interpretations for later viewing
- Dataset views that give zero-copy modified views into large datasets
### Longer term
- Callbacks and the ability to build interactive applications with Rerun
- For example: UI for tweaking configs, custom data annotation tools, etc
- Official ROS2 bridge
- Extensibility of all parts of the stack
- Data format stability
# Log and Ingest
In this section we'll log and visualize our first non-trivial dataset, putting many of Rerun's core concepts and features to use.
In a few lines of code, we'll go from a blank sheet to something you don't see every day: an animated, interactive, DNA-shaped abacus:
This guide aims to go wide instead of deep.
There are links to other doc pages where you can learn more about specific topics.
The complete code listings for this tutorial live alongside the Rerun source tree:
[Python](https://github.com/rerun-io/rerun/tree/latest/examples/python/dna/dna.py),
[Rust](https://github.com/rerun-io/rerun/tree/latest/examples/rust/dna/src/main.rs),
[C++](https://github.com/rerun-io/rerun/tree/latest/examples/cpp/dna/main.cpp).
## Prerequisites
Before starting, make sure you've [installed the SDK](https://rerun.io/docs/getting-started/install-rerun.md) and [set up a project](https://rerun.io/docs/getting-started/project-setup.md) for your language of choice.
## Initializing the SDK
Create a new file (or project), import the relevant utilities from your language's SDK, and initialize a recording. Initialization names the recording with a stable [`ApplicationId`](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md), then spawns a [Rerun Viewer](https://rerun.io/docs/reference/viewer/overview.md) and connects the recording to it:
```python
from math import tau
import numpy as np
import rerun as rr
from rerun.utilities import bounce_lerp, build_color_spiral
```
```python
rr.init("rerun_example_dna_abacus", spawn=True)
```
A stable `ApplicationId` will make the Viewer retain its UI state across runs for this specific dataset, which makes our lives much easier as we iterate.
By default, `spawn` will start a Viewer in another process and automatically pipe the data through. There are other ways to send data to a Viewer (covered at the end of this section), but the spawn default works great as we experiment.
## Logging our first points
The core structure of our DNA-looking shape can easily be described using two point clouds shaped like spirals:
```python
points1, colors1 = build_color_spiral(NUM_POINTS)
points2, colors2 = build_color_spiral(NUM_POINTS, angular_offset=tau * 0.5)
rr.log(
"dna/structure/left", rr.Points3D(points1, colors=colors1, radii=0.08)
)
rr.log(
"dna/structure/right", rr.Points3D(points2, colors=colors2, radii=0.08)
)
```
Run your program and you should now see this scene in the viewer.
If the Viewer was still running, Rerun will simply connect to this existing session and replace the data with this new [_recording_](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md).
_This is a good time to make yourself familiar with the viewer: try interacting with the scene and exploring the different menus._
_Checkout the [Viewer Walkthrough](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md) and [viewer reference](https://rerun.io/docs/reference/viewer/overview.md) for a complete tour of the viewer's capabilities._
## Under the hood
This tiny snippet of code actually holds much more than meets the eyeβ¦
### Archetypes
The easiest way to log geometric primitives is to use the SDK's `log` method with one of the built-in archetype classes (such as `Points3D` here). Archetypes take care of building batches of components that are recognized and correctly displayed by the Rerun viewer.
### Components
Under the hood, the Rerun SDK logs individual _components_ like positions, colors, and radii. Archetypes are just one high-level, convenient way of building such collections of components. For advanced use cases, it's possible to add custom components to archetypes, or even log entirely custom sets of components, bypassing archetypes altogether.
For more information on how the Rerun data model works, refer to our section on [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md). For supplying your own components, see [Use custom data](https://rerun.io/docs/howto/logging-and-ingestion/custom-data.md).
### Entities & hierarchies
Note the two strings we're passing in: `"dna/structure/left"` & `"dna/structure/right"`.
These are [_entity paths_](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md), which uniquely identify each entity in our scene. Every entity is made up of a path and one or more components.
[Entity paths typically form a hierarchy](https://rerun.io/docs/concepts/logging-and-ingestion/entity-path.md) which plays an important role in how data is visualized and transformed (as we shall soon see).
### Component batches
One final observation: notice how we're logging a whole batch of points and colors all at once.
[Component batches](https://rerun.io/docs/concepts/logging-and-ingestion/batches.md) are first-class citizens in Rerun and come with all sorts of performance benefits and dedicated features.
You're looking at one of these dedicated features right now: notice how we're only logging a single radius for all these points, yet somehow it applies to all of them. We call this _clamping_.
---
A _lot_ is happening in these two simple function calls.
Good news is: once you've digested all of the above, logging any other entity will simply be more of the same. In fact, let's go ahead and log everything else in the scene now.
## Adding the missing pieces
We can represent the scaffolding using a batch of 3D line strips:
```python
rr.log(
"dna/structure/scaffolding",
rr.LineStrips3D(
np.stack((points1, points2), axis=1), colors=[128, 128, 128]
),
)
```
Which only leaves the beads:
```python
offsets = np.random.rand(NUM_POINTS)
beads = [
bounce_lerp(points1[n], points2[n], offsets[n])
for n in range(NUM_POINTS)
]
colors = [
[int(bounce_lerp(80, 230, offsets[n] * 2))] for n in range(NUM_POINTS)
]
rr.log(
"dna/structure/scaffolding/beads",
rr.Points3D(beads, radii=0.06, colors=np.repeat(colors, 3, axis=-1)),
)
```
Once again, although we are getting fancier with our array manipulations, there is nothing new here: it's all about populating archetypes and feeding them to the Rerun API.
## Animating the beads
### Introducing time
Up until this point, we've completely set aside one of the core concepts of Rerun: [Time and Timelines](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md).
Even so, if you look at your [Timeline View](https://rerun.io/docs/reference/viewer/timeline.md) right now, you'll notice that Rerun has kept track of time on your behalf anyway by memorizing when each log call occurred.
Unfortunately, the logging time isn't particularly helpful to us in this case: we can't have our beads animate depending on the logging time, else they would move at different speeds depending on the performance of the logging process!
For that, we need to introduce our own custom timeline that uses a deterministic clock which we control.
Rerun has rich support for time: whether you want concurrent or disjoint timelines, out-of-order insertions or even data that lives _outside_ the timeline(s). You will find a lot of flexibility in there.
Replace the section that logs the beads with a loop that logs them at different timestamps:
```python
for i in range(400):
time = i * 0.01
rr.set_time("stable_time", duration=time)
times = np.repeat(time, NUM_POINTS) + time_offsets
beads = [
bounce_lerp(points1[n], points2[n], times[n])
for n in range(NUM_POINTS)
]
colors = [
[int(bounce_lerp(80, 230, times[n] * 2))] for n in range(NUM_POINTS)
]
rr.log(
"dna/structure/scaffolding/beads",
rr.Points3D(
beads, radii=0.06, colors=np.repeat(colors, 3, axis=-1)
),
)
```
A call to `set_time` (or `set_duration_secs` in Rust / `set_time_duration` in C++) creates our new `Timeline` and makes sure that any logging calls that follow get assigned that time.
You can add as many timelines and timestamps as you want when logging data.
> [!WARNING]
> If you run this code as is, the result will be⦠surprising: the beads are animating as expected, but everything we've logged until that point is gone!
Enterβ¦
### Latest-at semantics
That's because the Rerun Viewer has switched to displaying your custom timeline by default, but the original data was only logged to the _default_ timeline (called `log_time`).
To fix this, set the custom timeline to time zero before logging the original structure:
```python
rr.set_time("stable_time", duration=0)
```
This fix actually introduces yet another very important concept in Rerun: "latest-at" semantics.
Notice how entities `"dna/structure/left"` & `"dna/structure/right"` have only ever been logged at time zero, and yet they are still visible when querying times far beyond that point.
_Rerun always reasons in terms of "latest" data: for a given entity, it retrieves all of its most recent components at a given time._
## Transforming space
There's only one thing left: our original scene had the abacus rotate along its principal axis.
As was the case with time, (hierarchical) space transformations are first-class citizens in Rerun.
Now it's just a matter of combining the two: we need to log the transform of the scaffolding at each timestamp.
Either expand the previous loop to include logging transforms or simply add a second loop like this:
```python
for i in range(400):
time = i * 0.01
rr.set_time("stable_time", duration=time)
rr.log(
"dna/structure",
rr.Transform3D(
rotation=rr.RotationAxisAngle(
axis=[0, 0, 1], radians=time / 4.0 * tau
)
),
)
```
Voila!
## Other ways of logging & visualizing data
`spawn` is great when you're experimenting on a single machine like we did in this tutorial, but what if the logging happens on, for example, a headless computer?
Rerun offers several solutions for such use cases.
### Logging data over the network
At any time, you can start a Rerun Viewer by running `rerun`. This Viewer is in fact a server that's ready to accept data over gRPC (it's listening on `0.0.0.0:9876` by default).
On the logger side, replace the `spawn` call from above with a `connect_grpc` call to send data to any gRPC address:
```python
"""DNA-abacus example, connecting to a separately-running viewer over gRPC."""
import rerun as rr
rr.init("rerun_example_dna_abacus")
rr.connect_grpc() # connect to the viewer running at the default URL
# β¦ log data as in the spawn-based example β¦
```
Run `rerun --help` for more options.
### Saving & loading to/from RRD files
Sometimes, sending data over the network is not an option. Maybe you'd like to share the data, attach it to a bug report, etc.
Rerun has you covered: each SDK exposes a `save` method (Python: [`rr.save`](https://ref.rerun.io/docs/python/stable/common/initialization_functions/#rerun.save), Rust: [`RecordingStream::save`](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.save), C++: [`RecordingStream::save`](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a555a7940a076c93d951de5b139d14918)) that streams all logged data to disk. View the resulting file with `rerun path/to/recording.rrd`.
You can also save a recording (or a portion of it) as you're visualizing it, directly from the viewer.
### RRD file backwards compatibility
RRD files saved with Rerun 0.23 or later can be opened with a newer Rerun version.
For more details and potential limitations, please refer to [our blog post](https://rerun.io/blog/release-0.23).
> [!WARNING]
> At the moment, we only guarantee compatibility across adjacent minor versions (e.g. Rerun 0.24 can open RRDs from 0.23).
### Rust-only: showing the Viewer in-process
The Rust SDK can host the Viewer directly inside your application via [`rerun::native_viewer::show`](https://docs.rs/rerun/latest/rerun/native_viewer/fn.show.html), which expects a complete recording from memory rather than a live stream. This requires enabling the `native_viewer` feature in `Cargo.toml`. The Viewer blocks the main thread until closed; see the Rust API docs for details.
## Closing
This closes our whirlwind tour of logging with Rerun. We've barely scratched the surface of what's possible, but this should have hopefully given you plenty of pointers to start experimenting.
As a next step, browse through our [example gallery](https://rerun.io/examples) for some more realistic example use-cases, browse the [Types](https://rerun.io/docs/reference/types.md) section for more simple examples of how to use the main datatypes, or dig deeper into [querying your logged data](https://rerun.io/docs/getting-started/data-out.md).
## Opening files
You can also open existing files (RRD, MCAP, images, video, point clouds, etc.) directly with the Viewer β see [Opening files](https://rerun.io/docs/getting-started/data-in/open-any-file.md).
# Visualize
This guide will familiarize you with the basics of using the Rerun Viewer with an example dataset. By the end you should be comfortable with the following topics:
- [Prerequisites](#prerequisites)
- [Launching an example](#launching-an-example)
- [The Viewer panels](#the-viewer-panels)
- [Exploring data](#exploring-data)
- [Hover and selection](#hover-and-selection)
- [Rotate, zoom, and pan](#rotate-zoom-and-pan)
- [Navigating the timeline](#navigating-the-timeline)
- [Selecting different timelines](#selecting-different-timelines)
- [Conclusion](#conclusion)
- [Up next](#up-next)
Here is a preview of the dataset that we will be working with:
The demo uses the output of the [COLMAP](https://colmap.github.io/) structure-from-motion pipeline on a small dataset.
Familiarity with structure-from-motion algorithms is not a prerequisite for following the guide. All you need to know is
that at a very high level, COLMAP processes a series of images, and by tracking identifiable "keypoints" from frame to
frame, it is able to reconstruct both a sparse representation of the scene as well as the positions of the camera used
to take the images.
## Prerequisites
Although the Rerun SDK is available in both Python and Rust, this walkthrough makes use the Python installation. Even if
you plan to use Rerun with Rust, we still recommend having a Rerun Python environment available for quick
experimentation and working with examples. You can either follow the [Log and Ingest tutorial](https://rerun.io/docs/getting-started/data-in.md) or simply run:
```bash
pip install rerun-sdk
```
You can also find `rerun-sdk` on [`conda`](https://github.com/conda-forge/rerun-sdk-feedstock).
## Launching an example
If you have already followed the Python Quickstart you may have already check the "Helix" integrated example. This time, we will use the "Structure from Motion" example.
Start by running the viewer:
```bash
$ rerun
```
_Note: If this is your first time launching Rerun you will see a notification about the Rerun anonymous data usage
policy. Rerun collects anonymous usage data to help improve the SDK, though you may choose to opt out if you would
like._
This will bring you the Rerun viewer's Welcome screen:
From there you can chose the "Structure from Motion" example. A window that looks like this will appear:
Depending on your display size, the panels may have a different arrangements. Further in this guide you will learn how you can change that.
## The Viewer panels
This window has five main sections:
- [Viewport](https://rerun.io/docs/reference/viewer/viewport.md) (center): Displays the rendered views for your session.
- [Recordings panel](https://rerun.io/docs/concepts/logging-and-ingestion/recordings.md) (top left): Lists loaded recordings and their applications, and allows navigation back to the welcome screen.
- [Blueprint panel](https://rerun.io/docs/reference/viewer/blueprints.md) (below Recordings): Controls the different views.
- [Selection panel](https://rerun.io/docs/reference/viewer/selection.md) (right): Shows detailed information and configuration for selected items.
- [Timeline panel](https://rerun.io/docs/reference/viewer/timeline.md) (bottom): Controls the current point in time being viewed.
Each of the three sides has a button in the upper-right corner. Click these to show or hide the corresponding panels.
There are several ways to rearrange the viewer layout to your liking: through the Viewer [user interface](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md),
via the [Blueprint API](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md), or by [loading an .rbl file](https://rerun.io/docs/getting-started/configure-the-viewer/navigating-the-viewer.md).
## Exploring data
In Rerun, data is modeled using [entities](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) (essentially objects) that contain batches of [components](https://rerun.io/docs/reference/types/components.md)
that change over time. Each entity is identified by an entity path, which uses a hierarchical syntax to represent relationships between entities.
Let's explore an example of this hierarchy in our scene:
- `/camera/image/keypoints` is an entity stream that contains 2 component streams (`Color`, `Position2D`)
of the [Points2D archetype](https://rerun.io/docs/reference/types/archetypes/points2d.md),
representing point clouds that were detected and tracked in images.
- The images themselves are represented by the parent entity `/camera/image`. This entity consist of 6 components: 4 form an [Image archetype](https://rerun.io/docs/reference/types/archetypes/image.md),
while the remaining 2 correspond to a [pinhole projection](https://rerun.io/docs/reference/types/archetypes/pinhole.md). The images are captures by the camera, and a pinhole projection defines the relationship between 2D and 3D space.
- Both the images and pinhole projection are hierarchically dependent on the camera's position, which is described by the `/camera` entity. This entity includes a series of transforms that together form a [Transform3D archetype](https://rerun.io/docs/reference/types/archetypes/transform3d.md).
The hierarchy of logged entity streams and their component streams is found under `Streams` in the Timeline panel. A similar list appears in the `Blueprint` panel, but the key difference is that the Blueprint panel focuses on how data is arranged and visualized in the Viewport, while the Streams panel shows when and what events were logged. In other words, an entity may be logged once but displayed in multiple views.
Visualizations can also be customized per each view using [Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) in the Selection panel. In the screenshot below, the same entity `keypoints` is displayed in different colors: yellow and magenta. This is reflected in Selection > Visualizers > Points2D > Color, where yellow is an overridden value, even though the logged color value was different.
### Hover and selection
You can easily identify which entity mentions and visual representations refer to the same entities across different panels by seeing them simultaneously highlighted in the UI. Hovering over an entity will
display a popup with additional information about its content. Clicking on it will reveal more details in the [Selection panel](https://rerun.io/docs/reference/viewer/selection.md).
Try each of the following:
- Hover over the image to see a zoomed-in preview
- Click on the point cloud to select the whole cloud
- With the point cloud selected, hover and click individual points
### Rotate, zoom, and pan
Clicking and dragging the contents of any view will move it. You can rotate 3D views, or pan 2D views and plots. You can
also zoom using ctrl+scrollwheel or pinch gestures on a trackpad. Most views can be restored to their default state by
double-clicking somewhere in the view. Every view has a "?" icon in the upper right hand corner. You can always mouse
over this icon to find out more information about the specific view.
Try each of the following:
- Drag the camera image and zoom in on one of the stickers
- Rotate the 3D point cloud
- Right-click and drag a rectangle to see a zoomed-in region of the plot
- Double-click in each of the views to return them to default
## Navigating the timeline
If you look at the Timeline panel at the bottom of the window, you will see a series of white dots. Each of those dots
represents a piece of data that was logged at a different point in time. In fact, if you hover over the dot, the context popup will give you more information about
the specific thing that was logged.
There are several ways to navigate through the timeline:
- Move the time indicator by dragging it to a different point on the timeline.
You can also click on the frame number and manually type the desired frame.
- Adjust the playback speed, and for index-based timelines, you can also modify the number of frames per second to specify how indices relate to time.
- Use the play, pause, step, and loop controls to playback Rerun data, similar to how you would with a video file.
Try out the following:
- Use the arrow buttons (or Arrow keys on your keyboard) to step forward and backwards by a single frame
- Click play to watch the data update on its own
- Hit space bar to stop and start the playback
- Hold shift and drag in the timeline to select a region
- Toggle the loop button to playback on a loop of either the whole recording or just the selection
### Selecting different timelines
The current view of timeline is showing the data organized by the _frame number_ at which it was logged. Using frame
numbers can be a helpful way to synchronize things that may not have been logged at precisely the same time. However,
it's possible to also view the data in the specific order that it was logged. Click on the drop-down that says "frame"
and switch it to "log_time." If you zoom in on the timeline (using ctrl+scrollwheel), you can see that these events were
all logged at slightly different times.
Feel free to spend a bit of time looking at the data across the different timelines. When you are done, switch back
to the "frame" timeline and double-click the timeline panel to reset it to the default range.
One thing to notice is there is a gap in the timeline in the "frame" view. This dataset is actually missing a few
frames, and the timeline view of frames makes this easy to spot. This highlights the importance of applying meaningful
timestamps to your data as you log it. You also aren't limited to frame and log_time. Rerun lets you define your own
timelines however you would like. You can read more about timelines [here](https://rerun.io/docs/concepts/logging-and-ingestion/timelines.md).
## Conclusion
That brings us to the end of this walkthrough. To recap, you have learned how to:
- Install the `rerun-sdk` pypi package.
- Run the Rerun Viewer using the `rerun` command.
- Open the examples integrated in the viewer.
- Work with the [Blueprint](https://rerun.io/docs/reference/viewer/blueprints.md), [Selection](https://rerun.io/docs/reference/viewer/selection.md) and [Timeline](https://rerun.io/docs/reference/viewer/timeline.md) panels.
- Rearrange view layouts.
- Explore data through hover and selection.
- Change the time selection.
- Switch between different timelines.
Again, if you ran into any issues following this guide, please don't hesitate to [open an issue](https://github.com/rerun-io/rerun/issues/new/choose).
### Up next
- [Get started](https://rerun.io/docs/getting-started/data-in) by writing a program to log data with the Rerun SDK.
- Explore other [examples of using Rerun](https://rerun.io/examples).
- Consult the [concept overview](https://rerun.io/docs/concepts.md) for more context on the ideas covered here.
# Train
This page walks through streaming Rerun recordings directly into a PyTorch `DataLoader`, without an intermediate export step, using the bundled [LeRobot ACT training example](https://github.com/rerun-io/rerun/tree/main/examples/python/dataloader) end-to-end.
For an explanation of the dataloader API itself β windowed action chunks, GOP-aware video decoding, DDP partitioning β see [Train PyTorch models with Rerun](https://rerun.io/docs/howto/train.md).
> [!NOTE]
> The `rerun.experimental.dataloader` module is provisional and will change between releases.
## Run the example
The example trains a [LeRobot ACT](https://tonyzhaozh.github.io/aloha/) policy on the [`rerun/so101-pick-and-place`](https://huggingface.co/datasets/rerun/so101-pick-and-place) dataset from HuggingFace.
### 1. Grab the example
Sparse-checkout just the example directory, without the rest of the Rerun repo:
```bash
git clone --filter=blob:none --sparse https://github.com/rerun-io/rerun.git
cd rerun
git sparse-checkout set examples/python/dataloader
cd examples/python/dataloader
```
### 2. Install
The example has its own `uv` project because LeRobot pins an incompatible `rerun-sdk`.
The additional arguments to uv sync allow you to run just this example without the full rerun repo setup.
```bash
uv sync --no-sources --no-dev
```
If you have the full Rerun monorepo checked out and want to develop against your local Rerun build, run instead:
```bash
RERUN_ALLOW_MISSING_BIN=1 uv sync
uv pip install ../../../rerun_py/rerun_dev_fixup
```
### 3. Start a catalog server
In a separate terminal:
```bash
rerun server
```
### 4. Prepare and register the dataset
Downloads the dataset from HuggingFace, splits it into per-episode RRDs, and registers them with the catalog:
```bash
uv run python prepare_dataset.py
```
### 5. Train
```bash
uv run python train.py
```
The script streams batches from the catalog, trains an ACT policy for a few epochs, and saves a checkpoint to `act_checkpoint/`.
## References
- [Example source](https://github.com/rerun-io/rerun/tree/main/examples/python/dataloader) β `prepare_dataset.py` and `train.py`
- [`rerun/so101-pick-and-place`](https://huggingface.co/datasets/rerun/so101-pick-and-place) β LeRobot dataset on HuggingFace
- [Train PyTorch models with Rerun](https://rerun.io/docs/howto/train.md) β full how-to: windowing, video decoding, iterable vs. map style, DDP
# Query and Transform
At its core, Rerun is a database. The OSS server is our small-scale in-memory parallel to our commercial cloud offering.
In this three-part guide, we explore a query workflow by implementing an "open jaw detector" on top of our [face tracking example](https://rerun.io/examples/video-image/face_tracking). This process is split into three steps:
1. [Explore a recording with the dataframe view](https://rerun.io/docs/getting-started/data-out/explore-as-dataframe.md)
2. [Export the dataframe](https://rerun.io/docs/getting-started/data-out/export-dataframe.md)
3. [Analyze the data and send back the results](https://rerun.io/docs/getting-started/data-out/analyze-and-send.md)
> [!NOTE]
> This guide uses the popular [Pandas](https://pandas.pydata.org) dataframe package. The same concept however applies for alternative dataframe packages such as [Polars](https://pola.rs) or using [Datafusion](https://datafusion.apache.org/python/) directly.
If you just want to see the final result, jump to the [complete script](https://rerun.io/docs/getting-started/data-out/analyze-and-send.md) at the end of the third section.
# Install Rerun
Choose what you want to install:
- [Python](https://rerun.io/docs/getting-started/install-rerun/python.md) β the Python SDK (includes the Viewer)
- [C++](https://rerun.io/docs/getting-started/install-rerun/cpp.md) β the C++ SDK
- [Rust](https://rerun.io/docs/getting-started/install-rerun/rust.md) β the Rust SDK
- [Viewer](https://rerun.io/docs/getting-started/install-rerun/viewer.md) β the standalone Rerun Viewer application
If you run into any issues, check the [Troubleshooting](https://rerun.io/docs/getting-started/install-rerun/troubleshooting.md) guide.
# Set up a project
After [installing the SDK](https://rerun.io/docs/getting-started/install-rerun.md) for your language, set up a project that depends on Rerun. Pick your language:
- [Python](https://rerun.io/docs/getting-started/project-setup/python.md)
- [C++](https://rerun.io/docs/getting-started/project-setup/cpp.md)
- [Rust](https://rerun.io/docs/getting-started/project-setup/rust.md)
Once your project is set up, the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial walks through your first non-trivial recording.
# Set up a C++ project
You should have already [installed the C++ SDK](https://rerun.io/docs/getting-started/install-rerun/cpp.md).
We assume you have a working C++ toolchain and are using CMake to build your project.
For this project we will let Rerun download and build [Apache Arrow](https://arrow.apache.org/)'s C++ library itself.
To learn more about how Rerun's CMake script can be configured, see [CMake Setup in Detail](https://ref.rerun.io/docs/cpp/stable/md__2home_2runner_2work_2rerun_2rerun_2rerun__cpp_2cmake__setup__in__detail.html) in the C++ reference documentation.
## Setting up your CMakeLists.txt
A minimal `CMakeLists.txt` looks like this:
```cmake
cmake_minimum_required(VERSION 3.16...3.27)
project(example_project LANGUAGES CXX)
add_executable(example_project main.cpp)
# Download the rerun_sdk
include(FetchContent)
FetchContent_Declare(rerun_sdk URL
https://github.com/rerun-io/rerun/releases/latest/download/rerun_cpp_sdk.zip)
FetchContent_MakeAvailable(rerun_sdk)
# Link against rerun_sdk.
target_link_libraries(example_project PRIVATE rerun_sdk)
```
Note that Rerun requires at least C++17. Depending on the SDK will automatically ensure that C++17 or newer is enabled.
## Includes
To use Rerun all you need to include is `rerun.hpp`:
```cpp
#include
```
## Building
```bash
cmake -B build
cmake --build build -j
./build/example_project
```
You're now ready to follow the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# Set up a Python project
You should have already [installed the Python SDK](https://rerun.io/docs/getting-started/install-rerun/python.md).
A Python project doesn't require any setup beyond having `rerun-sdk` available.
Open your editor of choice, create a new file, and import Rerun:
```python
import rerun as rr
```
You're now ready to follow the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# Set up a Rust project
You should have already [installed the Rust SDK](https://rerun.io/docs/getting-started/install-rerun/rust.md).
If you haven't already, start a new project with `cargo new` and add the `rerun` dependency:
```bash
cargo new my_project
cd my_project
cargo add rerun
```
You're now ready to follow the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# C++ SDK
If you're using CMake you can add the SDK to your project using `FetchContent`:
```cmake
include(FetchContent)
FetchContent_Declare(rerun_sdk URL
https://github.com/rerun-io/rerun/releases/latest/download/rerun_cpp_sdk.zip)
FetchContent_MakeAvailable(rerun_sdk)
```
For more details see [Build & Distribution](https://ref.rerun.io/docs/cpp/stable/index.html#autotoc_md8) in the C++ reference documentation.
You'll additionally need to install the [Viewer](https://rerun.io/docs/getting-started/install-rerun/viewer.md).
## Next steps
[Set up a C++ project](https://rerun.io/docs/getting-started/project-setup/cpp.md), then walk through the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# Viewer
The [Viewer](https://rerun.io/docs/reference/viewer/overview.md) can be installed independent of the library language you're using.
Make sure that your library version matches the version of the Viewer you're using, because [our data format is not yet stable across different versions](https://github.com/rerun-io/rerun/issues/6410).
There are many ways to install the viewer. Please pick whatever works best for your setup:
- Download `rerun-cli` for your platform from the [GitHub Release artifacts](https://github.com/rerun-io/rerun/releases/latest/).
- Via Cargo
- `cargo binstall rerun-cli` - download binaries via [`cargo binstall`](https://github.com/cargo-bins/cargo-binstall)
- `cargo install rerun-cli --locked` - build it from source (this requires Rust 1.95+)
- Via Snap (_community maintained_)
- `snap install rerun` - download the viewer from the [Store](https://snapcraft.io/rerun).
- Together with the Rerun [Python SDK](https://rerun.io/docs/getting-started/install-rerun/python.md):
- `pip3 install rerun-sdk` - download it via pip
- `conda install -c conda-forge rerun-sdk` - download via Conda
- `pixi global install rerun-sdk` - download it via [Pixi](https://pixi.sh/latest/)
In any case you should be able to run `rerun` afterwards to start the Viewer.
You'll be welcomed by an overview page that allows you to jump into some examples.
If you're facing any difficulties, don't hesitate to [open an issue](https://github.com/rerun-io/rerun/issues/new/choose) or [join the Discord server](https://discord.gg/PXtCgFBSmH).
The Rerun Viewer has built-in support for opening many kinds of files, and can be [extended to open any other file type](https://rerun.io/docs/getting-started/data-in/open-any-file.md) without needing to modify the Rerun codebase itself.
# Troubleshooting
You can set `RUST_LOG=debug` before running to get some verbose logging output.
If you run into any issues don't hesitate to [open a ticket](https://github.com/rerun-io/rerun/issues/new/choose)
or [join our Discord](https://discord.gg/Gcm8BbTaAj).
## Running on Linux
Rerun should work out-of-the-box on Mac and Windows, but on Linux you need to first run:
```sh
sudo apt-get -y install \
libclang-dev \
libatk-bridge2.0 \
libfontconfig1-dev \
libfreetype6-dev \
libglib2.0-dev \
libgtk-3-dev \
libssl-dev \
libxcb-render0-dev \
libxcb-shape0-dev \
libxcb-xfixes0-dev \
libxkbcommon-dev \
patchelf
```
On Fedora Rawhide you need to run:
```sh
sudo dnf install \
clang \
clang-devel \
clang-tools-extra \
libxcb-devel \
libxkbcommon-devel \
openssl-devel \
pkg-config
```
[TODO(#1250)](https://github.com/rerun-io/rerun/issues/1250): Running with the wayland window manager
sometimes causes Rerun to crash. Try unsetting the wayland display (`unset WAYLAND_DISPLAY` or `WAYLAND_DISPLAY= `) as a workaround.
## Running on WSL2 (Ubuntu)
WSL's graphics drivers won't work out of the box and you'll have to update to a more recent version.
To install the latest stable version of the mesa Vulkan drivers run:
```sh
sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt-get update
sudo apt-get install -y mesa-vulkan-drivers
```
Since the Mesa driver on WSL dispatches to the Windows host graphics driver, it is important to keep the Windows drivers up-to-date as well.
For example, [line rendering issues](https://github.com/rerun-io/rerun/issues/6749) have been observed when running from WSL with an
outdated AMD driver on the Windows host.
On Ubuntu 24 [issues with Wayland](https://github.com/rerun-io/rerun/issues/6748) have been observed.
To mitigate this install `libxkbcommon-x11`
```
sudo apt install libxkbcommon-x11-0
```
And unset the wayland display either by `unset WAYLAND_DISPLAY` or `WAYLAND_DISPLAY= `.
## `pip install` issues
If you see the following when running `pip install rerun-sdk` or `pip install rerun-notebook` on a supported platform:
```sh
ERROR: Could not find a version that satisfies the requirement rerun-sdk (from versions: none)
ERROR: No matching distribution found for rerun-sdk
```
Then this is likely because you're running a version of pip that is too old.
You can check the version of pip with `pip --version`.
If you're running a version of pip 20 or older, you should upgrade it with `pip install --upgrade pip`.
> [!WARNING]
> Depending on your system configuration this may upgrade the pip installation aliased by `pip3` instead of `pip`.
## Startup issues
If Rerun is having trouble starting, you can try resetting its memory with:
```
rerun reset
```
## Graphics issues
Make sure to keep your graphics drivers updated.
[Wgpu](https://github.com/gfx-rs/wgpu) (the graphics API we use) maintains a list of
[known driver issues](https://github.com/gfx-rs/wgpu/wiki/Known-Driver-Issues) and workarounds for them.
The configuration we use for wgpu can be influenced in the following ways:
- pass `--renderer=` on startup: `` must be one of `vulkan`, `metal` or `gl` for native and
either `webgl` or `webgpu` for the web viewer (see also `--web-viewer` argument).
Naturally, support depends on your OS. The default backend is `vulkan` everywhere except on Mac where we use `metal`.
On the web we prefer WebGPU and fall back automatically to WebGL if no support for WebGPU was detected.
- For instance, you can try `rerun --renderer=gl` or for the web viewer respectively `rerun --web-viewer --renderer=webgl`.
- Alternatively, for the native viewer you can also use the `WGPU_BACKEND` environment variable with the above values.
- The web viewer is configured by the `renderer=` url argument, e.g. [https://rerun.io/viewer?renderer=webgl]
- `WGPU_POWER_PREF`: Overwrites the power setting used for choosing a graphics adapter, must be `high` or `low`. (Default is `high`)
We recommend setting these only if you're asked to try them or know what you're doing,
since we don't support all of these settings equally well.
### Multiple GPUs
When using Wgpu's Vulkan backend (the default on Windows & Linux) on a computer that has both integrated and dedicated GPUs, a lot of issues can arise from Vulkan either picking the "wrong" GPU at runtime, or even simply from the fact that this choice conflicts with other driver picking technologies (e.g. NVIDIA Optimus).
In both cases, forcing Vulkan to pick either the integrated or discrete GPU (try both!) using the [`VK_ICD_FILENAMES`](https://vulkan.lunarg.com/doc/view/latest/mac/LoaderDriverInterface.html#user-content-driver-discovery) environment variable might help with crashes, artifacts and bad performance. E.g.:
- Force the Intel integrated GPU:
- Linux: `export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel.json`.
- Force the discrete Nvidia GPU:
- Linux: `export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia.json`.
- Windows: `set VK_ICD_FILENAMES=\windows\system32\nv-vk64.json`.
## Video stuttering
On some browsers the default video decoder may cause stuttering.
This has been for instance observed with Chrome 129 on Windows.
To mitigate these issues, you can try to specify software decoding.
This can be configured from the viewer's option menu. Alternatively, you can also override this setting on startup:
* for the web viewer pass `&video_decoder=prefer_software` as a url parameter
* for the native viewer & for starting the web viewer via command line (`--web-viewer` argument), pass `--video-decoder=prefer_software`
For more information about video decoding, see also the reference page on [video](https://rerun.io/docs/concepts/logging-and-ingestion/video.md).
# Python SDK
The Python SDK includes both the SDK and the Viewer, so you're ready to go with a single install:
- `pip install rerun-sdk` via pip
- `conda install -c conda-forge rerun-sdk` via Conda
Conda always comes with support for all features but if using pip you may need to specify optional features:
- `pip install rerun-sdk[notebook]` for the embedded notebook tools
- `pip install rerun-sdk[catalog]` for the query api tools
- `pip install rerun-sdk[dataloader]` for model training tools
## Next steps
[Set up a Python project](https://rerun.io/docs/getting-started/project-setup/python.md), then walk through the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# Rust SDK
Add the [Rerun crate](https://crates.io/crates/rerun) using `cargo add rerun`.
You'll additionally need to install the [Viewer](https://rerun.io/docs/getting-started/install-rerun/viewer.md).
## Next steps
[Set up a Rust project](https://rerun.io/docs/getting-started/project-setup/rust.md), then walk through the [Log and Ingest](https://rerun.io/docs/getting-started/data-in.md) tutorial.
# Export the dataframe
In the [previous section](https://rerun.io/docs/getting-started/data-out/explore-as-dataframe.md), we explored some face tracking data using the dataframe view. In this section, we will see how we can use the dataframe API of the Rerun SDK to export the same data into a [Pandas](https://pandas.pydata.org) dataframe to further inspect and process it.
## Load the recording
The dataframe SDK loads data from an .RRD file.
The first step is thus to save the recording as RRD, which can be done from the Rerun menu:
We can then load the recording in a Python script as follows:
First perform the necessary imports,
```python
from __future__ import annotations
from pathlib import Path
import numpy as np
import rerun as rr
```
then launch the server to load the recording
```python
server = rr.server.Server(datasets={"tutorial": [example_rrd]})
client = rr.catalog.CatalogClient(server.url())
```
## Query the data
Once we loaded a recording, we can query it to extract some data. Here is how it is done:
```python
dataset = client.get_dataset("tutorial")
df = dataset.filter_contents("/blendshapes/0/jawOpen").reader(index="frame_nr")
```
A lot is happening here, let's go step by step:
1. We first create a _view_ into the recording. The view specifies which content we want to use (in this case the `"/blendshapes/0/jawOpen"` entity). The view defines a subset of all the data contained in the recording where each row has a unique value for the index.
2. In order to perform queries a view must become a dataframe. We use the `reader()` call to specify this transformation where we specify our index (timeline) of interest.
3. The object returned by `reader()` is a [`datafusion.Dataframe`](https://datafusion.apache.org/python/autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame).
[DataFusion](https://datafusion.apache.org/python/) provides a pythonic dataframe interface to your data as well as [SQL](https://datafusion.apache.org/python/user-guide/sql.html) querying.
## Create a Pandas dataframe
Before exploring the data further, let's convert the table to a Pandas dataframe:
```python
pd_df = df.to_pandas()
```
## Inspect the dataframe
Let's have a first look at this dataframe:
```python
print(df)
```
Here is the result:
```
frame_nr frame_time log_tick log_time /blendshapes/0/jawOpen:Scalars:scalars
0 0 1970-01-01 00:00:00.000 34 2024-10-13 08:26:46.819571 [0.03306490555405617]
1 1 1970-01-01 00:00:00.040 92 2024-10-13 08:26:46.866358 [0.03812221810221672]
2 2 1970-01-01 00:00:00.080 150 2024-10-13 08:26:46.899699 [0.027743922546505928]
3 3 1970-01-01 00:00:00.120 208 2024-10-13 08:26:46.934704 [0.024137917906045914]
4 4 1970-01-01 00:00:00.160 266 2024-10-13 08:26:46.967762 [0.022867577150464058]
.. ... ... ... ... ...
409 409 1970-01-01 00:00:16.360 21903 2024-10-13 08:27:01.619732 [0.07283800840377808]
410 410 1970-01-01 00:00:16.400 21961 2024-10-13 08:27:01.656455 [0.07037288695573807]
411 411 1970-01-01 00:00:16.440 22019 2024-10-13 08:27:01.689784 [0.07556036114692688]
412 412 1970-01-01 00:00:16.480 22077 2024-10-13 08:27:01.722971 [0.06996039301156998]
413 413 1970-01-01 00:00:16.520 22135 2024-10-13 08:27:01.757358 [0.07366073131561279]
[414 rows x 5 columns]
```
We can make several observations from this output:
- The first four columns are timeline columns. These are the various timelines the data is logged to in this recording.
- The last column is named `/blendshapes/0/jawOpen:Scalars:scalars`. This is what we call a _component column_, and it corresponds to the [Scalar](https://rerun.io/docs/reference/types/components/scalar.md) component logged to the `/blendshapes/0/jawOpen` entity.
- Each row in the `/blendshapes/0/jawOpen:Scalar` column consists of a _list_ of (typically one) scalar.
This last point may come as a surprise but is a consequence of Rerun's data model where components are always stored as arrays. This enables, for example, to log an entire point cloud using the [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md) archetype under a single entity and at a single timestamp.
Let's explore this further, recalling that, in our recording, no face was detected at around frame #170:
```python
print(pd_df["/blendshapes/0/jawOpen:Scalars:scalars"][160:180])
```
Here is the result:
```
160 [0.0397215373814106]
161 [0.037685077637434006]
162 [0.0402931347489357]
163 [0.04329492896795273]
164 [0.0394592322409153]
165 [0.020853394642472267]
166 []
167 []
168 []
169 []
170 []
171 []
172 []
173 []
174 []
175 []
176 []
177 []
178 []
179 []
Name: /blendshapes/0/jawOpen:Scalars:scalars, dtype: object
```
We note that the data contains empty lists when no face is detected. When the blendshapes entities are [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)ed, this happens for the corresponding timestamps and all further timestamps until a new value is logged.
While this data representation is in general useful, a flat floating point representation with `NaN` for missing values is typically more convenient for scalar data. This is achieved using the [`explode()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.explode.html) method:
```python
pd_df["jawOpen"] = (
pd_df["/blendshapes/0/jawOpen:Scalars:scalars"].explode().astype(float)
)
print(pd_df["jawOpen"][160:180])
```
Here is the result:
```
160 0.039722
161 0.037685
162 0.040293
163 0.043295
164 0.039459
165 0.020853
166 NaN
167 NaN
168 NaN
169 NaN
170 NaN
171 NaN
172 NaN
173 NaN
174 NaN
175 NaN
176 NaN
177 NaN
178 NaN
179 NaN
Name: jawOpen, dtype: float64
```
This confirms that the newly created `"jawOpen"` column now contains regular, 64-bit float numbers, and missing values are represented by NaNs.
> [!NOTE]
> Should you want to filter out the NaNs, you may use the [`dropna()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.dropna.html) method.
## Next steps
With this, we are ready to analyze the data and log back the result to the Rerun viewer, which is covered in the [next section](https://rerun.io/docs/getting-started/data-out/analyze-and-send.md) of this guide.
# Analyze the data and send back the results
In the previous sections, we explored our data and exported it to a Pandas dataframe. In this section, we will analyze the data to extract a "jaw open state" signal and send it back to the viewer.
## Analyze the data
We already identified that thresholding the `jawOpen` signal at 0.15 is all we need to produce a binary "jaw open state" signal.
In the [previous section](https://rerun.io/docs/getting-started/data-out/export-dataframe.md), we prepared a flat, floating point column with the signal of interest called `"jawOpen"`. Let's add a boolean column to our Pandas dataframe to hold our jaw open state:
```python
# compute the mouth state
pd_df["jawOpenState"] = pd_df["jawOpen"] > 0.15
```
## Send the result back to the viewer
The first step is to initialize the logging SDK targeting the same recording we just analyzed.
This requires using the exact same application ID and recording ID.
By using the same identifiers, we're appending new data to an existing recording.
If the recording is currently open in the viewer and listening for new connections, the new data will appear in the active session in real time.
```python
rr.init(application_id, recording_id=dataset.segment_ids()[0])
rr.connect_grpc()
```
> [!NOTE]
> When automating data analysis, it is typically preferable to log the results to an distinct RRD file next to the source RRD (using `rr.save()`). In such a situation, it is also valid to use the same app ID and recording ID. This allows opening both the source and result RRDs in the viewer, which will display data from both files under the same recording.
We will send our jaw open state data in two forms:
1. As a standalone [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md) component, to hold the raw data.
2. As a [`Text`](https://rerun.io/docs/reference/types/components/text.md) component on the existing bounding box entity, such that we obtain a textual representation of the state in the visualization.
Here is how to send the data as a scalar:
```python
rr.send_columns(
"/jaw_open_state",
indexes=[rr.TimeColumn("frame_nr", sequence=pd_df["frame_nr"])],
columns=rr.Scalars.columns(scalars=pd_df["jawOpenState"]),
)
```
We use the [`rr.send_column()`](https://rerun.io/docs/howto/logging-and-ingestion/send-columns.md) API to efficiently send the entire column of data in a single batch.
Next, let's send the same data as `Text` component:
```python
target_entity = "/video/detector/faces/0/bbox"
rr.log(target_entity, rr.Boxes2D.from_fields(show_labels=True), static=True)
rr.send_columns(
target_entity,
indexes=[rr.TimeColumn("frame_nr", sequence=pd_df["frame_nr"])],
columns=rr.Boxes2D.columns(
labels=np.where(pd_df["jawOpenState"], "OPEN", "CLOSE")
),
)
```
Here we first log the [`ShowLabel`](https://rerun.io/docs/reference/types/components/show_labels.md) component as static to enable the display of the label. Then, we use `rr.send_column()` again to send an entire batch of text labels. We use [`np.where()`](https://numpy.org/doc/stable/reference/generated/numpy.where.html) to produce a label matching the state for each timestamp.
### Final result
With some adjustments to the viewer blueprint, we obtain the following result:
The OPEN/CLOSE label is displayed along the bounding box on the 2D view, and the `/jaw_open_state` signal is visible in both the timeseries and dataframe views.
### Complete script
Here is the complete script used by this guide to load data, analyze it, and send the result back:
```python
# region: imports
from __future__ import annotations
from pathlib import Path
import numpy as np
import rerun as rr
# endregion: imports
# ----------------------------------------------------------------------------
# Load and prepare the data
repo_root = Path(__file__).parent.parent.parent.parent.parent
example_rrd = (
repo_root / "tests" / "assets" / "rrd" / "examples" / "face_tracking.rrd"
)
assert example_rrd.exists(), f"Example RRD not found at {example_rrd}"
# region: launch_server
server = rr.server.Server(datasets={"tutorial": [example_rrd]})
client = rr.catalog.CatalogClient(server.url())
# endregion: launch_server
# query the recording into a pandas dataframe
# region: query_data
dataset = client.get_dataset("tutorial")
df = dataset.filter_contents("/blendshapes/0/jawOpen").reader(index="frame_nr")
# endregion: query_data
# region: to_pandas
pd_df = df.to_pandas()
# endregion: to_pandas
# region: print_frames
print(pd_df["/blendshapes/0/jawOpen:Scalars:scalars"][160:180])
# endregion: print_frames
# convert the "jawOpen" column to a flat list of floats
print(pd_df)
# region: explode_jaw
pd_df["jawOpen"] = (
pd_df["/blendshapes/0/jawOpen:Scalars:scalars"].explode().astype(float)
)
print(pd_df["jawOpen"][160:180])
# endregion: explode_jaw
# ----------------------------------------------------------------------------
# Analyze the data
# region: filter_jaw
# compute the mouth state
pd_df["jawOpenState"] = pd_df["jawOpen"] > 0.15
# endregion: filter_jaw
# ----------------------------------------------------------------------------
# Log the data back to the viewer
application_id = (
rr.experimental.RrdReader(example_rrd).recordings()[0].application_id
)
# Connect to the viewer
# region: connect_viewer
rr.init(application_id, recording_id=dataset.segment_ids()[0])
rr.connect_grpc()
# endregion: connect_viewer
# log the jaw open state signal as a scalar
# region: send_columns
rr.send_columns(
"/jaw_open_state",
indexes=[rr.TimeColumn("frame_nr", sequence=pd_df["frame_nr"])],
columns=rr.Scalars.columns(scalars=pd_df["jawOpenState"]),
)
# endregion: send_columns
# log a `Label` component to the face bounding box entity
# region: log_labels
target_entity = "/video/detector/faces/0/bbox"
rr.log(target_entity, rr.Boxes2D.from_fields(show_labels=True), static=True)
rr.send_columns(
target_entity,
indexes=[rr.TimeColumn("frame_nr", sequence=pd_df["frame_nr"])],
columns=rr.Boxes2D.columns(
labels=np.where(pd_df["jawOpenState"], "OPEN", "CLOSE")
),
)
# endregion: log_labels
```
# Explore a recording with the dataframe view
In this first part of the guide, we run the [face tracking example](https://rerun.io/examples/video-image/face_tracking) and explore the data in the viewer.
## Create a recording
The first step is to create a recording in the viewer using the face tracking example. Check the [face tracking installation instruction](https://rerun.io/examples/video-image/face_tracking#run-the-code) for more information on how to run this example.
Here is such a recording:
A person's face is visible and being tracked. Their jaws occasionally open and close. In the middle of the recording, the face is also temporarily hidden and no longer tracked.
## Explore the data
Amongst other things, the [MediaPipe Face Landmark](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker) package used by the face tracking example outputs so-called blendshapes signals, which provide information on various aspects of the face expression. These signals are logged under the `/blendshapes` root entity by the face tracking example.
One signal, `jawOpen` (logged under the `/blendshapes/0/jawOpen` entity as a [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md) component), is of particular interest for our purpose. Let's inspect it further using a timeseries view:
This signal indeed seems to jump from approximately 0.0 to 0.5 whenever the jaws are open. We also notice a discontinuity in the middle of the recording. This is due to the blendshapes being [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)ed when no face is detected.
Let's create a dataframe view to further inspect the data:
Here is how this view is configured:
- Its content is set to `/blendshapes/0/jawOpen`. As a result, the table only contains columns pertaining to that entity (along with any timeline(s)). For this entity, a single column exists in the table, corresponding to entity's single component (a `Scalar`).
- The `frame_nr` timeline is used as index for the table. This means that the table will contain one row for each distinct value of `frame_nr` for which data is available.
- The rows can further be filtered by time range. In this case, we keep the default "infinite" boundaries, so no filtering is applied.
- The dataframe view has other advanced features which we are not using here, including filtering rows based on the existence of data for a given column, or filling empty cells with latest-at data.
Now, let's look at the actual data as represented in the above screenshot. At around frame #140, the jaws are open, and, accordingly, the `jawOpen` signal has values around 0.55. Shortly after, they close again and the signal decreases to below 0.1. Then, the signal becomes empty. This happens in rows corresponding to the period of time when the face cannot be tracked and all the signals are cleared.
## Next steps
Our exploration of the data in the viewer so far provided us with two important pieces of information useful to implement the jaw open detector.
First, we identified that the `Scalar` value contained in `/blendshapes/0/jawOpen` contains relevant data. In particular, thresholding this signal with a value of 0.15 should provide us with a closed/opened jaw state binary indicator.
Then, we explored the numerical data in a dataframe view. Importantly, the way we configured this view for our needs informs us on how to query the recording from code such as to obtain the correct output.
From there, our next step is to query the recording and extract the data as a Pandas dataframe in Python. This is covered in the [next section](https://rerun.io/docs/getting-started/data-out/export-dataframe.md) of this guide.
# Navigating the Viewer
This page will walk you through the basics of navigating the Rerun Viewer.
By default, the Rerun Viewer uses heuristics to automatically determine an appropriate layout for your data. However, you'll often want precise control over how your data is displayed. Blueprints give you complete control over the Viewer's layout and configuration. For a conceptual understanding of blueprints, see [Blueprints](https://rerun.io/docs/concepts/visualization/blueprints.md).
This guide covers three complementary ways to work with the viewer:
- **[Interactive configuration](#interactive-configuration)**: Modify layouts directly in the Viewer UI
- **[Save and load blueprint files](#save-and-load-blueprint-files)**: Share layouts using `.rbl` files
- **[Programmatic blueprints](#programmatic-blueprints)**: Control layouts from code
## Interactive configuration
The Rerun Viewer is fully configurable through its UI, making it easy to experiment with different layouts.
### Viewer overview
The Viewer consists of:
- **Viewport** (center): Contains your views, arranged in containers
- **Blueprint Panel** (left): Shows the visual tree of your blueprint structure
- **Selection Panel** (right): Displays properties of the selected element
- **Time Panel** (bottom): Controls timeline playback and navigation
The blueprint defines what appears in the viewport. All changes you make to the viewport are actually changes to the blueprint.
### Configuring the view hierarchy
The viewport contains views arranged hierarchically using containers. Containers come in four types:
- **Horizontal**: Arranges views side-by-side
- **Vertical**: Stacks views top-to-bottom
- **Grid**: Organizes views in a grid layout
- **Tabs**: Shows views in tabs (only one visible at a time)
#### Add new containers or views
Click the "+" button at the top of the blueprint panel to add containers or views.
If a container (or the viewport) is selected, a "+" button also appears in the selection panel.
#### Rearrange views and containers
Drag and drop items in the blueprint panel to reorganize the hierarchy. You can also drag views directly in the viewport using their title tabs.
#### Show, hide, or remove elements
Use the eye icon to show or hide any container, view, or entity:
Use the "-" button to permanently remove an element:
#### Rename views and containers
Select a view or container and edit its name at the top of the selection panel.
#### Change container type
Select a container and change its type using the dropdown in the selection panel.
#### Using context menus
Right-click on any element in the blueprint panel for quick access to common operations:
Context menus support multi-selection (Ctrl+click or Cmd+click), enabling bulk operations like removing multiple views at once.
### Configuring view content
Each view displays data based on its entity query. You can modify what appears in a view interactively.
#### Show or hide entities
Use the eye icon next to any entity to control its visibility.
#### Remove entities from views
Click the "-" button next to an entity to remove it from the view.
#### Using the query editor
With a view selected, click "Edit" next to the entity query in the selection panel to visually add or remove entities.
#### Creating views from entities
Select one or more entities (in existing views or in the time panel's streams), right-click, and choose "Add to new view" from the context menu.
The view's origin will automatically be set based on the selected data.
### Overriding visualizers and components
Select an entity within a view to control which visualizers are used and override component values.
When selecting a view, you can also set default component values that apply when no value has been logged.
See [Visualizers and Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) for detailed information.
---
## Save and load blueprint files
Once you've configured your layout, you can save it as a blueprint file (`.rbl`) to reuse across sessions or share with your team.
### Saving a blueprint
To save your current blueprint, go to the file menu and choose "Save blueprintβ¦":
Blueprint files are small, portable, and can be version-controlled alongside your code.
### Loading a blueprint
Load a blueprint file using "Openβ¦" from the file menu, or simply drag and drop the `.rbl` file into the Viewer.
> [!IMPORTANT]
> The blueprint's Application ID must match the Application ID of your recording. Blueprints are bound to specific Application IDs to ensure they work with compatible data structures. See [Application IDs](https://rerun.io/docs/concepts/visualization/blueprints.md) for more details.
### Sharing blueprints
Blueprint files make it easy to ensure everyone on your team views data consistently:
1. Configure your ideal layout interactively
2. Save the blueprint to a `.rbl` file
3. Commit the file to your repository
4. Team members load the blueprint when viewing recordings with the same Application ID
This is particularly valuable for:
- **Debugging sessions**: Share the exact layout needed to diagnose specific issues
- **Presentations**: Ensure consistent visualization across demos
- **Data analysis**: Standardize views for comparing results
---
## Programmatic blueprints
You can also define blueprints entirely from code using the Blueprint API. This is ideal for creating reproducible layouts, generating views dynamically based on your data, or integrating blueprint configuration into your logging pipeline.
For a complete guide, see [Build a blueprint programmatically](https://rerun.io/docs/howto/visualization/build-a-blueprint-programmatically.md) and the [Blueprints concept page](https://rerun.io/docs/concepts/visualization/blueprints.md).
---
## Next steps
- **Explore view types**: Check the [View Type Reference](https://rerun.io/docs/reference/types/views/) to see all available views and their configuration options
- **Learn about overrides**: See [Visualizers and Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) for per-entity customization
- **API Reference**: Browse the complete [Blueprint API](https://ref.rerun.io/docs/python/stable/common/blueprint_apis/) for programmatic control
# Opening files
The Rerun Viewer and SDK have built-in support for opening many kinds of files, and can be [extended](https://rerun.io/docs/concepts/logging-and-ingestion/importers/overview.md) to support any other file type without needing to modify the Rerun codebase itself.
The Viewer can load files in 3 different ways:
- via CLI arguments (e.g. `rerun myfile.jpeg`),
- using drag-and-drop,
- using the open dialog in the Rerun Viewer.
All these file loading methods support loading a single file, many files at once (e.g. `rerun myfiles/*`), or even folders.
> [!WARNING]
> Drag-and-drop of folders does [not yet work](https://github.com/rerun-io/rerun/issues/4528) on the web version of the Rerun Viewer.
The following data types have built-in support in the Rerun Viewer and SDK:
- Native Rerun files: `rrd`
- 3D models: `gltf`, `glb`, `obj`, `stl`
- Images: `avif`, `bmp`, `dds`, `exr`, `farbfeld`, `ff`, `gif`, `hdr`, `ico`, `jpeg`, `jpg`, `pam`, `pbm`, `pgm`, `png`, `ppm`, `tga`, `tif`, `tiff`, `webp`
- Point clouds: `ply`
- Text files: `md`, `txt`
- [LeRobot](https://huggingface.co/docs/lerobot/index) datasets: `directory`
With the exception of `rrd` files that can be streamed from an HTTP URL (e.g. `rerun https://demo.rerun.io/version/latest/examples/dna/data.rrd`), we only support loading files from the local filesystem for now, with [plans to make this generic over any URI and protocol in the future](https://github.com/rerun-io/rerun/issues/4525).
## Logging file contents from the SDK
To log the contents of a file from the SDK you can use the `log_file_from_path` and `log_file_from_contents` methods ([C++](https://ref.rerun.io/docs/cpp/stable/classrerun_1_1RecordingStream.html#a8f253422a7adc2a19b89d1538c05bcac), [Python](https://ref.rerun.io/docs/python/stable/common/other_classes_and_functions/#rerun.log_file_from_path), [Rust](https://docs.rs/rerun/latest/rerun/struct.RecordingStream.html#method.log_file_from_path)) and the associated examples ([C++](https://github.com/rerun-io/rerun/blob/main/examples/cpp/log_file/main.cpp), [Python](https://github.com/rerun-io/rerun/blob/main/examples/python/log_file/log_file.py), [Rust](https://github.com/rerun-io/rerun/blob/main/examples/rust/log_file/src/main.rs)).
> [!NOTE]
> When calling these APIs from the SDK, the data will be loaded by the process running the SDK, not the Viewer!
```python
import sys
import rerun as rr
rr.init("rerun_example_log_file", spawn=True)
rr.log_file_from_path(sys.argv[1])
```
# Types
Rerun has 3 levels of types that can be used in all SDK languages:
* [**Archetypes**](https://rerun.io/docs/reference/types/archetypes.md) - high level bundles of components.
* [**Components**](https://rerun.io/docs/reference/types/components.md) - the base unit of logging data.
* [**Data types**](https://rerun.io/docs/reference/types/datatypes.md) - that make up the individual components.
To get an overview of what the Rerun [Viewer](https://rerun.io/docs/reference/viewer/overview.md) can show, start at [**Archetypes**](https://rerun.io/docs/reference/types/archetypes.md).
For more information on the relationship between **archetypes** and **components**, check out the concept page
on [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md).
# About
To learn more about Rerun, the company, visit our Website at [https://www.rerun.io/](https://www.rerun.io/).
Code & License
--------------
The Rerun SDK & Viewer are open source, all code is available on [GitHub](https://github.com/rerun-io/rerun/) and open for contributions.
Licensing is permissive, the project is dual licensed under [MIT](https://github.com/rerun-io/rerun/blob/main/LICENSE-MIT) & [Apache 2.0](https://github.com/rerun-io/rerun/blob/main/LICENSE-APACHE).
Under the hood
--------------
The software is almost entirely written in [Rust](https://www.rust-lang.org/), a modern, fast and safe programming language.
If you're curious about why we love Rust, checkout our [blog](https://www.rerun.io/blog/why-rust), where we talk about some of the reasons.
We depend on a number of third party libraries, most notably:
* [Apache Arrow](https://arrow.apache.org/) for our data store
* [wgpu](https://wgpu.rs/) for rendering
* [egui](https://github.com/emilk/egui) for UI
* [PyO3](https://github.com/PyO3/pyo3) for Python bindings
If you want to learn more about the different parts of the SDK & Viewer and how they work, check out
[this architecture overview](https://github.com/rerun-io/rerun/blob/latest/ARCHITECTURE.md)
for an introduction.
# Datastore compaction
The Rerun datastore continuously compacts data as it comes in, in order find a sweet spot between ingestion speed, query performance and memory overhead.
The compaction is triggered by both number of rows and number of bytes thresholds, whichever happens to trigger first.
This is very similar to, and has many parallels with, the [micro-batching mechanism running on the SDK side](https://rerun.io/docs/reference/sdk/micro-batching.md).
You can configure these thresholds using the following environment variables:
#### RERUN_CHUNK_MAX_BYTES
Sets the threshold, in bytes, after which a `Chunk` cannot be compacted any further.
Defaults to `RERUN_CHUNK_MAX_BYTES=4194304` (4MiB).
#### RERUN_CHUNK_MAX_ROWS
Sets the threshold, in rows, after which a `Chunk` cannot be compacted any further.
Defaults to `RERUN_CHUNK_MAX_ROWS=4096`.
#### RERUN_CHUNK_MAX_ROWS_IF_UNSORTED
Sets the threshold, in rows, after which a `Chunk` cannot be compacted any further.
Applies specifically to _non_ time-sorted chunks, which can be slower to query.
Defaults to `RERUN_CHUNK_MAX_ROWS=1024`.
# β¨οΈ CLI manual
## rerun
The Rerun command-line interface:
* Spawn viewers to visualize Rerun recordings and other supported formats.
* Start a gRPC server to share recordings over the network, on native or web.
* Inspect, edit and filter Rerun recordings.
**Usage**: ` rerun [OPTIONS] [URL_OR_PATHS]β¦ [COMMAND]`
**Commands**
* `analytics`: Configure the behavior of our analytics.
* `auth`: Authentication with the redap.
* `download`: Download recordings and save them as .rrd files.
* `man`: Generates the Rerun CLI manual (markdown).
* `mcap`: Manipulate the contents of .mcap files.
* `viewer-mcp`: Run an MCP server that controls a running Rerun Viewer.
* `reset`: Reset the memory of the Rerun Viewer.
* `rrd`: Manipulate the contents of .rrd and .rbl files.
* `server`: In-memory Rerun data server.
**Arguments**
* ``
> Any combination of:
> - A gRPC url to a Rerun server
> - A path to a Rerun .rrd recording
> - A path to a Rerun .rbl blueprint
> - An HTTP(S) URL to an .rrd or .rbl file to load
> - A path to an image or mesh, or any other file that Rerun can load (see https://www.rerun.io/docs/concepts/logging-and-ingestion/importers/overview)
>
> If no arguments are given, a server will be hosted which a Rerun SDK can connect to.
**Options**
* `--bind `
> What bind address IP to use.
>
> `::` will listen on all interfaces, IPv6 and IPv4.
>
> [Default: `0.0.0.0`]
* `--memory-limit `
> An upper limit on how much memory the Rerun Viewer should use.
> When this limit is reached, Rerun will drop the oldest data.
> Example: `16GB` or `50%` (of system total).
> You can also set this in the settings panel.
* `--server-memory-limit `
> An upper limit on how much memory the gRPC server (`--serve-web`) should use.
> The server buffers log messages for the benefit of late-arriving viewers.
> When this limit is reached, Rerun will drop the oldest data.
> Example: `16GB` or `50%` (of system total).
>
> [Default: `1GiB`]
* `--newest-first `
> If true, play back the most recent data first when new clients connect.
>
> [Default: `false`]
* `--cors-allow-origin `
> Additional origin patterns allowed to make CORS requests to the gRPC server.
>
> Use this when hosting a custom viewer on a different domain. Patterns are matched against the full Origin header (e.g. `https://example.com:8080`), using glob-style matching where `*` matches any sequence of characters. Can be specified multiple times.
>
> Examples: `--cors-allow-origin "https://*.example.com"` `--cors-allow-origin "https://example.com:8080"` `--cors-allow-origin "https://example.com:*"`
* `--persist-state `
> Whether the Rerun Viewer should persist the state of the viewer to disk.
> When persisted, the state will be stored at the following locations:
> - Linux: `/home/UserName/.local/share/rerun`
> - macOS: `/Users/UserName/Library/Application Support/rerun`
> - Windows: `C:\Users\UserName\AppData\Roaming\rerun`
>
> [Default: `true`]
* `--port `
> What port do we listen to for SDKs to connect to over gRPC.
>
> Use `auto` to always start a new viewer with a free port if the default is taken.
>
> [Default: `9876`]
* `--new `
> Alias for `--port auto`. Always start a new viewer.
>
> If the port is already in use, a free port will be picked automatically.
>
> [Default: `false`]
* `--profile `
> Start with the puffin profiler running.
>
> [Default: `false`]
* `--save `
> Stream incoming log events to an .rrd file at the given path.
* `--screenshot-to `
> Take a screenshot of the app and quit. We use this to generate screenshots of our examples. Useful together with `--window-size`.
* `--serve-web `
> This will host a web-viewer over HTTP, and a gRPC server, unless one or more URIs are provided that can be viewed directly in the web viewer.
>
> If started, the web server will act like a proxy, listening for incoming connections from logging SDKs, and forwarding it to Rerun viewers.
>
> [Default: `false`]
* `--serve-grpc `
> This will host a gRPC server.
>
> The server will act like a proxy, listening for incoming connections from logging SDKs, and forwarding it to Rerun viewers.
>
> [Default: `false`]
* `--connect `
> Do not attempt to start a new server, instead try to connect to an existing one.
>
> Optionally accepts a URL to a gRPC server.
>
> The scheme must be one of `rerun://`, `rerun+http://`, or `rerun+https://`, and the pathname must be `/proxy`.
>
> The default is `rerun+http://127.0.0.1:9876/proxy`.
* `--expect-data-soon `
> This is a hint that we expect a recording to stream in very soon.
>
> This is set by the `spawn()` method in our logging SDK.
>
> The viewer will respond by fading in the welcome screen, instead of showing it directly. This ensures that it won't blink for a few frames before switching to the recording.
>
> [Default: `false`]
* `-j, --threads `
> The number of compute threads to use.
>
> If zero, the same number of threads as the number of cores will be used. If negative, will use that much fewer threads than cores.
>
> Rerun will still use some additional threads for I/O.
>
> [Default: `-2`]
* `--version `
> Print version and quit.
>
> [Default: `false`]
* `--web-viewer `
> Start the viewer in the browser (instead of locally).
>
> Requires Rerun to have been compiled with the `web_viewer` feature.
>
> This implies `--serve-web`.
>
> [Default: `false`]
* `--web-viewer-port `
> What port do we listen to for hosting the web viewer over HTTP. A port of 0 will pick a random port.
>
> [Default: `9090`]
* `--hide-welcome-screen `
> Hide the normal Rerun welcome screen.
>
> [Default: `false`]
* `--detach-process `
> Detach Rerun Viewer process from the application process.
>
> [Default: `false`]
* `--headless `
> Run the viewer in headless mode (no OS window).
>
> The viewer is driven by an offscreen `egui_kittest` harness, while the gRPC server keeps running so SDK clients can still log data and request screenshots via `save_screenshot`.
>
> [Default: `false`]
* `--window-size `
> Set the screen resolution (in logical points), e.g. "1920x1080". Useful together with `--screenshot-to`.
* `--renderer `
> Override the default graphics backend and for a specific one instead.
>
> When using `--web-viewer` this should be one of: `webgpu`, `webgl`.
>
> When starting a native viewer instead this should be one of:
>
> * `vulkan` (Linux & Windows only)
>
> * `gl` (Linux & Windows only)
>
> * `metal` (macOS only)
* `--video-decoder `
> Overwrites hardware acceleration option for video decoding.
>
> By default uses the last provided setting, which is `auto` if never configured.
>
> Depending on the decoder backend, these settings are merely hints and may be ignored.
> However, they can be useful in some situations to work around issues.
>
> Possible values:
>
> * `auto`
> May use hardware acceleration if available and compatible with the codec.
>
> * `prefer_software`
> Should use a software decoder even if hardware acceleration is available.
> If no software decoder is present, this may cause decoding to fail.
>
> * `prefer_hardware`
> Should use a hardware decoder.
> If no hardware decoder is present, this may cause decoding to fail.
* `--test-receive `
> Ingest data and then quit once the goodbye message has been received.
>
> Used for testing together with `RERUN_PANIC_ON_WARN=1`.
>
> Fails if no messages are received, or if no messages are received within a dozen or so seconds.
>
> [Default: `false`]
## rerun analytics
Configure the behavior of our analytics.
**Usage**: `rerun analytics `
**Commands**
* `details`: Prints extra information about analytics.
* `clear`: Deletes everything related to analytics.
* `email`: Associate an email address with the current user.
* `enable`: Enable analytics.
* `disable`: Disable analytics.
* `config`: Prints the current configuration.
## rerun analytics email
Associate an email address with the current user.
**Usage**: `rerun analytics email `
**Arguments**
* ``
## rerun auth
Authentication with the redap.
**Usage**: `rerun auth `
**Commands**
* `login`: Log into Rerun.
* `logout`: Log out of Rerun.
* `token`: Retrieve the stored access token.
* `generate-token`: Generate a fresh access token.
## rerun auth login
Log into Rerun.
This command opens a page in your default browser, allowing you to log in to Rerun Hub.
Once you've logged in, your credentials are stored on your machine.
To sign up, contact us through the form linked at .
**Usage**: `rerun auth login [OPTIONS]`
**Options**
* `--no-open-browser `
> Post a link instead of directly opening in the browser.
>
> [Default: `false`]
* `--force `
> Trigger the full login flow even if valid credentials already exist.
>
> [Default: `false`]
## rerun auth logout
Log out of Rerun.
This command clears the credentials stored on your machine and ends your session.
**Usage**: `rerun auth logout [OPTIONS]`
**Options**
* `--no-open-browser `
> Post a link instead of directly opening in the browser.
>
> [Default: `false`]
## rerun auth generate-token
Generate a fresh access token.
You can use this token to authorize requests to Rerun Hub.
It's closer to an API key than an access token, as it can be revoked before it expires.
**Usage**: `rerun auth generate-token [OPTIONS] --server --expiration `
**Options**
* `--server `
> Origin of the server to request the token from.
* `--expiration `
> Duration of the token, either in: - "human time", e.g. `1 day`, or - ISO 8601 duration format, e.g. `P1D`.
* `--permission `
> Which permission the token should have.
>
> [`read`, `read-write`]
>
> [Default: `read`]
## rerun download
Download recordings and save them as .rrd files.
Supports downloading from Rerun Hub as well as any other supported URI.
**Usage**: `rerun download [OPTIONS] β¦`
**Arguments**
* ``
> One or more URIs to download.
**Options**
* `-o, --output-dir `
> Override the output directory for the downloaded `.rrd` files.
>
> Defaults to the current working directory.
## rerun mcap
Manipulate the contents of .mcap files.
**Usage**: `rerun mcap `
**Commands**
* `convert`: Convert an .mcap file to an .rrd.
* `info`: Print timeline / sortedness diagnostics for an .mcap file.
## rerun mcap convert
Convert an .mcap file to an .rrd.
**Usage**: `rerun mcap convert [OPTIONS] `
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-o, --output `
> Path to write to. Writes to standard output if unspecified.
* `--application-id `
> If set, specifies the application id of the output.
* `-d, --decoder `
> Specifies which decoders to apply during conversion.
* `--disable-raw-fallback `
> Disable using the raw decoder as a fallback for unsupported channels. By default, channels that cannot be handled by semantic decoders (protobuf, ROS2) will be processed by the raw decoder.
>
> [Default: `false`]
* `--recording-id `
> If set, specifies the recording id of the output.
>
> When this flag is set and multiple input .rdd files are specified, blueprint activation commands will be dropped from the resulting output.
* `--timestamp-offset-ns `
> If set, an offset in nanoseconds to add to all timestamp timelines.
>
> This can be used to shift all timestamps of the MCAP file if they are not yet relative to the UNIX epoch.
>
> Duration and sequence timelines are not affected by this offset.
* `--timeline-type `
> The timeline type to use for timestamp timelines.
>
> "timestamp" (default) creates `TimestampNs` timelines (nanoseconds since Unix epoch). "duration" creates `DurationNs` timelines (nanosecond durations).
>
> [Default: `timestamp`]
* `-y, --include-topic-regex `
> Include only topics matching this regex (RE2 syntax). Repeatable.
>
> If omitted, all topics are included. Patterns are not implicitly anchored; use `^` / `$` if you need anchoring.
>
> Example: `-y "^/tf.*" -n ".*depth.*" -y "^/camera/(compressed|camera_info)$"`
* `-n, --exclude-topic-regex `
> Exclude topics matching this regex (RE2 syntax). Repeatable.
>
> Applied after includes: a topic is kept only if it matches an include (or no includes are set) AND matches no exclude.
* `--start-time `
> Inclusive lower bound on the raw MCAP `log_time`.
>
> Accepts Unix timestamps with a unit suffix (`ns`, `ms`, `s`, β¦), or an RFC 3339 timestamp. Bare integers are interpreted as nanoseconds.
>
> If set, only data within this time range gets converted.
* `--end-time `
> Exclusive upper bound on the raw MCAP `log_time`.
>
> Accepts Unix timestamps with a unit suffix (`ns`, `ms`, `s`, β¦), or an RFC 3339 timestamp. Bare integers are interpreted as nanoseconds.
>
> If set, only data within this time range gets converted.
* `--recover `
> Recover a missing or invalid MCAP summary in memory.
>
> This allows conversion of MCAP files that lack a footer (e.g. corrupted recordings).
>
> [Default: `false`]
## rerun mcap info
Print timeline / sortedness diagnostics for an .mcap file.
**Usage**: `rerun mcap info [OPTIONS] `
**Arguments**
* ``
> Path to the .mcap file to inspect.
**Options**
* `--full `
> Run the full `re_mcap` decoder pipeline.
>
> Surfaces timelines added by per-message decoders (e.g. `ros2_timestamp` from a ROS 2 `Header.stamp`). Without this flag only the raw MCAP-level timelines `message_log_time` / `message_publish_time` are inspected.
>
> [Default: `false`]
## rerun rrd
Manipulate the contents of .rrd and .rbl files.
**Usage**: `rerun rrd `
**Commands**
* `compare`: Compares the data between 2 .rrd files, returning a successful shell exit code if they match.
* `filter`: Filters out data from .rrd/.rbl files/streams, and writes the result to standard output.
* `merge`: Merges the contents of multiple .rrd/.rbl files/streams, and writes the result to standard output.
* `migrate`: Migrate one or more .rrd files to the newest Rerun version.
* `optimize`: Optimizes the contents of one or more .rrd/.rbl files/streams by compacting chunks, and writes the result to standard output.
* `print`: Print the contents of one or more .rrd/.rbl files/streams.
* `route`: Manipulates the metadata of log message streams without decoding the payloads.
* `split`: Optimally splits a recording on a specified timeline.
* `stats`: Compute important statistics for one or more .rrd/.rbl files/streams.
* `verify`: Verify the that the .rrd file can be loaded and correctly interpreted.
## rerun rrd compare
Compares the data between 2 .rrd files, returning a successful shell exit code if they match.
This ignores the `log_time` timeline.
**Usage**: `rerun rrd compare [OPTIONS] `
**Arguments**
* ``
* ``
**Options**
* `--unordered `
> If specified, the comparison will focus purely on semantics, ignoring order.
>
> The Rerun data model is itself unordered, and because many of the internal pipelines are asynchronous by nature, it is very easy to end up with semantically identical, but differently ordered data. In most cases, the distinction is irrelevant, and you'd rather the comparison succeeds.
>
> [Default: `false`]
* `--full-dump `
> If specified, dumps both .rrd files as tables.
>
> [Default: `false`]
* `--ignore-chunks-without-components `
> If specified, the comparison will ignore chunks without components.
>
> [Default: `false`]
* `--ignore-timeline `
> Timelines to ignore entirely during comparison (their presence, absence, and values).
>
> Useful when comparing recordings produced with different default-timeline settings, e.g. `--ignore-timeline log_tick` (which is opt-in). Can be specified multiple times.
## rerun rrd filter
Filters out data from .rrd/.rbl files/streams, and writes the result to standard output.
Reads from standard input if no paths are specified.
This will not affect the chunking of the data in any way.
Example: `rerun rrd filter --drop-timeline log_tick /my/recordings/*.rrd > output.rrd`
**Usage**: `rerun rrd filter [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-o, --output `
> Path to write to. Writes to standard output if unspecified.
* `--drop-timeline `
> Names of the timelines to be filtered out.
* `--drop-entity `
> Paths of the entities to be filtered out.
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
>
> [Default: `false`]
## rerun rrd merge
Merges the contents of multiple .rrd/.rbl files/streams, and writes the result to standard output.
Reads from standard input if no paths are specified.
β οΈ This will automatically migrate the data to the latest version of the RRD protocol, if needed. β οΈ
Example: `rerun rrd merge /my/recordings/*.rrd > output.rrd`
**Usage**: `rerun rrd merge [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-o, --output `
> Path to write to. Writes to standard output if unspecified.
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
>
> [Default: `false`]
## rerun rrd migrate
Migrate one or more .rrd files to the newest Rerun version.
Example: `rerun rrd migrate foo.rrd` Results in a `foo.backup.rrd` (copy of the old file) and a new `foo.rrd` (migrated).
**Usage**: `rerun rrd migrate [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to rrd files to migrate.
## rerun rrd optimize
Optimizes the contents of one or more .rrd/.rbl files/streams by compacting chunks, and writes the result to standard output.
Reads from standard input if no paths are specified.
If any input is a directory, the command switches to **directory mirror mode**: every `.rrd`/`.rbl` file under the input is optimized independently, and written to the output path while preserving the input folder structure. In this mode the output (`-o`) must be set and is treated as a directory root.
Uses the usual environment variables to control the compaction thresholds: `RERUN_CHUNK_MAX_ROWS`, `RERUN_CHUNK_MAX_ROWS_IF_UNSORTED`, `RERUN_CHUNK_MAX_BYTES`.
Unless explicit flags are passed, in which case they will override environment values.
Video stream chunks are also rebatched on GoP (keyframe) boundaries so that each chunk holds one or more complete GoPs. Pass `--no-rebatch-videos` to disable that.
β οΈ This will automatically migrate the data to the latest version of the RRD protocol, if needed. β οΈ
Examples:
* Optimize a single recording into one optimized file (`-o`): `rerun rrd optimize my.rrd -o my-compacted.rrd`
* Merge many recordings into one optimized file: `rerun rrd optimize --max-size 2MiB /my/recordings/*.rrd -o output.rrd`
* Pipe through stdin/stdout, overriding both row and size thresholds: `cat my.rrd | rerun rrd optimize --max-rows 4096 --max-size 2MiB > output.rrd`
* Directory mirror mode β optimize every `.rrd`/`.rbl` under a tree, preserving structure: `rerun rrd optimize --max-size 2MiB /my/recordings -o /my/recordings-compacted`
**Usage**: `rerun rrd optimize [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-o, --output `
> Path to write the optimized recording to.
>
> In single-file mode (the default), this is the output file path. If unspecified, the recording is written to standard output.
>
> In directory mirror mode (when any input is a directory), this must be set and is treated as the output directory root: the input folder structure is mirrored underneath it, with each `.rrd`/`.rbl` file optimized independently.
* `--profile `
> Optimization profile to start from.
>
> Per-knob flags and `RERUN_CHUNK_MAX_*` env vars override the profile's values. `RERUN_STORE_ENABLE_CHANGELOG` is ignored by this command β `rerun rrd optimize` is always headless.
>
> [Default: `object-store`]
* `--max-size `
> Threshold after which a Chunk cannot be compacted any further.
>
> Accepts a size string with a unit suffix, e.g. `2MiB`, `512KiB`, `1GB`, `1024B`. Both binary (`KiB`/`MiB`/`GiB`/`TiB`) and decimal (`kB`/`MB`/`GB`/`TB`) units are accepted.
>
> Overrides the profile's value and `RERUN_CHUNK_MAX_BYTES` if set.
* `--max-rows `
> What is the threshold, in rows, after which a Chunk cannot be compacted any further?
>
> Overrides the profile's value and `RERUN_CHUNK_MAX_ROWS` if set.
* `--max-rows-if-unsorted `
> What is the threshold, in rows, after which a Chunk cannot be compacted any further?
>
> This specifically applies to _non_ time-sorted chunks.
>
> Overrides the profile's value and `RERUN_CHUNK_MAX_ROWS_IF_UNSORTED` if set.
* `--num-pass `
> Configures the number of extra compaction passes to run on the data. Overrides the profile's value. Default per profile: 50.
>
> Compaction in Rerun is an iterative, convergent process: every single pass will improve the quality of the compaction (with diminishing returns), until it eventually converges into a stable state. The more passes, the better the compaction quality.
>
> Under the hood, you can think of it as a kind of clustering algorithm: every incoming chunk finds the most appropriate chunk to merge into, thereby creating a new cluster, which is itself just a bigger chunk. On the next pass, these new clustered chunks will themselves look for other clusters to merge into, yielding even bigger clusters, which again are also just chunks. And so on and so forth.
>
> If/When the data reaches a stable optimum, the computation will stop immediately, regardless of how many passes are left.
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
>
> [Default: `false`]
* `--no-rebatch-videos `
> Disable rebatching of video stream chunks to GoP (Group of Pictures) boundaries.
>
> By default, after compaction, video stream chunks are rebatched on GoP boundaries so that each chunk contains one or more complete GoPs. This flag disables that behavior.
>
> Note: GoP rebatching never splits a GoP across chunks, so streams with long keyframe intervals (e.g. 10+ seconds between I-frames) can produce chunks much larger than `--max-size`.
>
> [Default: `false`]
* `--fix-keyframe `
> Drop any user-supplied `VideoStream:is_keyframe` labels and re-derive them from the encoded samples.
>
> By default, `rrd optimize` validates user-supplied keyframe labels against the encoded samples and errors out if they disagree. Pass this flag to ignore the existing labels and unconditionally re-derive them.
>
> [Default: `false`]
* `--split-size-ratio `
> If set, split chunks so no two archetype groups sharing a chunk differ in byte size by more than this factor. Values should be `>= 1`; at `1.0`, every archetype is forced into its own chunk.
>
> This keeps "thick" columns (images, videos, blobs) out of the same chunk as "thin" columns (scalars, transforms, text), so the viewer can fetch just the thin data without dragging along the thick payload. Components belonging to the same archetype are always kept together.
>
> A good starting value is 10.0. If unset, the profile's value is used.
## rerun rrd print
Print the contents of one or more .rrd/.rbl files/streams.
Reads from standard input if no paths are specified.
Example: `rerun rrd print /my/recordings/*.rrd`
**Usage**: `rerun rrd print [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-v, --verbose `
> If set, print out table contents.
>
> This can be specified more than once to toggle more and more verbose levels (e.g. -vvv):
>
> * default: summary with short names.
>
> * `-v`: summary with fully-qualified names.
>
> * `-vv`: show all chunk metadata headers, keep the data hidden.
>
> * `-vvv`: show all chunk metadata headers as well as the data itself.
>
> [Default: `0`]
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
* `--migrate `
> Migrate chunks to latest version before printing?
* `--full-metadata `
> If true, includes `rerun.` prefixes on keys.
* `--entity `
> Show only chunks belonging to this entity.
* `--footers `
> If true, displays all the parsed footers at the end.
* `--footers-lod `
> The level of detail to use when printing footers. Higher is more detailed.
>
> * `0`: only chunk metadata columns
>
> * `1`: `0` + global timeline columns
>
> * `2`: `1` + everything else
>
> [Default: `0`]
* `--transposed `
> Transpose record batches before printing them?
## rerun rrd route
Manipulates the metadata of log message streams without decoding the payloads.
This can be used to combine multiple .rrd files into a single recording. Example: `rerun rrd route --recording-id my_recording /my/recordings/*.rrd > output.rrd`
Note: Because the payload of the messages is never decoded, no migration or verification will performed.
**Usage**: `rerun rrd route [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `-o, --output `
> Path to write to. Writes to standard output if unspecified.
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
>
> [Default: `false`]
* `--application-id `
> If set, specifies the application id of the output.
* `--recording-id `
> If set, specifies the recording id of the output.
>
> When this flag is set and multiple input .rdd files are specified, blueprint activation commands will be dropped from the resulting output.
## rerun rrd split
Optimally splits a recording on a specified timeline.
The sum of the generated splits will always exactly match the original recording.
Example: `rerun rrd split --output-dir ./splits --timeline log_tick --time 33 --time 66 ./my_video.rrd`
**Usage**: `rerun rrd split [OPTIONS] --output-dir --timeline `
**Arguments**
* ``
> Path to read from.
**Options**
* `-o, --output-dir `
> Path to the output directory. All generated RRD files will end up there.
* `--timeline `
> The timeline used to compute the splits.
>
> The other timelines will be kept in the output, which might or might not make sense depending on the density of the dataset. Use `--drop-unused-timelines` to discard them.
* `-t, --time `
> The timestamps at which to perform the splits. Incompatible with `--num-parts`/`-n`.
>
> There are always `number_of_times + 1` resulting splits.
>
> For example, given `-t 10 -t 20 -t 30`, this command will output 4 splits: [-inf:10), [10:20), [20:30), [30:+inf).
* `-n, --num-parts `
> The number of parts to split the recording into. Incompatible with `--time`/`-t`.
>
> There will be exactly that number of resulting splits. Each split will cover an equal time span in the timeline.
* `--recording-id `
> The recording ID prefix to be used for the output recordings.
>
> If left unspecified, the ID of the original recording, suffixed with a `-`, will be used as a prefix.
>
> Each split will use `` as their respective recording ID, where `i` is the index of the split.
* `--drop-unused-timelines `
> If true, timelines other than the one specified with `--timeline` will be discarded.
>
> [Default: `false`]
## rerun rrd stats
Compute important statistics for one or more .rrd/.rbl files/streams.
Reads from standard input if no paths are specified.
Example: `rerun rrd stats /my/recordings/*.rrd`
**Usage**: `rerun rrd stats [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `--no-decode `
> If set, the data will never be decoded.
>
> Statistics will be computed at the transport-level instead, which is more limited in terms of what can be computed, but also orders of magnitude faster.
>
> [Default: `false`]
* `--continue-on-error `
> If set, will try to proceed even in the face of IO and/or decoding errors in the input data.
>
> [Default: `true`]
## rerun rrd verify
Verify the that the .rrd file can be loaded and correctly interpreted.
Can be used to ensure that the current Rerun version can load the data.
**Usage**: `rerun rrd verify [OPTIONS] [PATH_TO_INPUT_RRDS]β¦`
**Arguments**
* ``
> Paths to read from. Reads from standard input if none are specified.
**Options**
* `--check-footers `
> If true, ensures that RRD footers are present and well formed.
>
> [Default: `true`]
## rerun server
In-memory Rerun data server.
**Usage**: `rerun server [OPTIONS]`
**Options**
* `--host `
> IP address to listen on.
>
> [Default: `0.0.0.0`]
* `-p, --port `
> Port to bind to.
>
> [Default: `51234`]
* `-d, --dataset <[NAME=]DIR_PATH>`
> Load a directory of RRD as dataset (can be specified multiple times). You can specify only a path or provide a name such as `-d my_dataset=./path/to/files`.
* `-t, --table <[NAME=]TABLE_PATH>`
> Load a lance file as a table (can be specified multiple times). You can specify only a path or provide a name such as `-t my_table=./path/to/table`.
* `--latency-ms `
> Artificial latency to add to each request (in milliseconds).
>
> [Default: `0`]
* `--bandwidth-limit `
> Artificial bandwidth limit for responses (e.g. '10MB' for 10 megabytes per second).
* `--cors-allow-origin `
> Additional origin patterns allowed to make cross-origin requests to the server (can be specified multiple times).
>
> By default, only `localhost`, `127.0.0.1`, and `rerun.io` are allowed. Patterns are matched against the full `Origin` header value, using glob-style matching where `*` matches any sequence of characters.
* `-V, --version `
> Print version.
# Datatypes
Data types are the lowest layer of the data model hierarchy. They are re-usable types used by the components.
* [`AbsoluteTimeRange`](https://rerun.io/docs/reference/types/datatypes/absolute_time_range.md): Two [`datatypes.TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int) describing a range of time.
* [`Angle`](https://rerun.io/docs/reference/types/datatypes/angle.md): Angle in radians.
* [`AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info.md): Annotation info annotating a class id or key-point id.
* [`Blob`](https://rerun.io/docs/reference/types/datatypes/blob.md): A binary blob of data.
* [`Bool`](https://rerun.io/docs/reference/types/datatypes/bool.md): A single boolean.
* [`ChannelCountPair`](https://rerun.io/docs/reference/types/datatypes/channel_count_pair.md): A pair representing a channel ID and its associated message count.
* [`ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype.md): The innermost datatype of an image.
* [`ClassDescription`](https://rerun.io/docs/reference/types/datatypes/class_description.md): The description of a semantic Class.
* [`ClassDescriptionMapElem`](https://rerun.io/docs/reference/types/datatypes/class_description_map_elem.md): A helper type for mapping [`datatypes.ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id)s to class descriptions.
* [`ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id.md): A 16-bit ID representing a type of semantic class.
* [`ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model.md): Specified what color components are present in an [`archetypes.Image`](https://rerun.io/docs/reference/types/archetypes/image).
* [`DVec2D`](https://rerun.io/docs/reference/types/datatypes/dvec2d.md): A double-precision vector in 2D space.
* [`EntityPath`](https://rerun.io/docs/reference/types/datatypes/entity_path.md): A path to an entity in the `ChunkStore`.
* [`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md): A single-precision 32-bit IEEE 754 floating point number.
* [`Float64`](https://rerun.io/docs/reference/types/datatypes/float64.md): A double-precision 64-bit IEEE 754 floating point number.
* [`IVec3D`](https://rerun.io/docs/reference/types/datatypes/ivec3d.md): An int32 vector in 3D space.
* [`ImageFormat`](https://rerun.io/docs/reference/types/datatypes/image_format.md): The metadata describing the contents of a [`components.ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer).
* [`KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id.md): A 16-bit ID representing a type of semantic keypoint within a class.
* [`KeypointPair`](https://rerun.io/docs/reference/types/datatypes/keypoint_pair.md): A connection between two [`datatypes.KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id)s.
* [`Mat3x3`](https://rerun.io/docs/reference/types/datatypes/mat3x3.md): A 3x3 Matrix.
* [`Mat4x4`](https://rerun.io/docs/reference/types/datatypes/mat4x4.md): A 4x4 Matrix.
* [`PixelFormat`](https://rerun.io/docs/reference/types/datatypes/pixel_format.md): Specifieds a particular format of an [`archetypes.Image`](https://rerun.io/docs/reference/types/archetypes/image).
* [`Plane3D`](https://rerun.io/docs/reference/types/datatypes/plane3d.md): An infinite 3D plane represented by a unit normal vector and a distance.
* [`Quaternion`](https://rerun.io/docs/reference/types/datatypes/quaternion.md): A Quaternion represented by 4 real numbers.
* [`Range1D`](https://rerun.io/docs/reference/types/datatypes/range1d.md): A 1D range, specifying a lower and upper bound.
* [`Range2D`](https://rerun.io/docs/reference/types/datatypes/range2d.md): An Axis-Aligned Bounding Box in 2D space, implemented as the minimum and maximum corners.
* [`Rgba32`](https://rerun.io/docs/reference/types/datatypes/rgba32.md): An RGBA color with unmultiplied/separate alpha, in sRGB gamma space with linear alpha.
* [`RotationAxisAngle`](https://rerun.io/docs/reference/types/datatypes/rotation_axis_angle.md): 3D rotation represented by a rotation around a given axis.
* [`TensorBuffer`](https://rerun.io/docs/reference/types/datatypes/tensor_buffer.md): The underlying storage for [`archetypes.Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor).
* [`TensorData`](https://rerun.io/docs/reference/types/datatypes/tensor_data.md): An N-dimensional array of numbers.
* [`TensorDimensionIndexSelection`](https://rerun.io/docs/reference/types/datatypes/tensor_dimension_index_selection.md): Indexing a specific tensor dimension.
* [`TensorDimensionSelection`](https://rerun.io/docs/reference/types/datatypes/tensor_dimension_selection.md): Selection of a single tensor dimension.
* [`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md): A 64-bit number describing either nanoseconds OR sequence numbers.
* [`TimeRange`](https://rerun.io/docs/reference/types/datatypes/time_range.md): Visible time range bounds for a specific timeline.
* [`TimeRangeBoundary`](https://rerun.io/docs/reference/types/datatypes/time_range_boundary.md): Left or right boundary of a time range.
* [`UInt16`](https://rerun.io/docs/reference/types/datatypes/uint16.md): A 16bit unsigned integer.
* [`UInt32`](https://rerun.io/docs/reference/types/datatypes/uint32.md): A 32bit unsigned integer.
* [`UInt64`](https://rerun.io/docs/reference/types/datatypes/uint64.md): A 64bit unsigned integer.
* [`UVec2D`](https://rerun.io/docs/reference/types/datatypes/uvec2d.md): A uint32 vector in 2D space.
* [`UVec3D`](https://rerun.io/docs/reference/types/datatypes/uvec3d.md): A uint32 vector in 3D space.
* [`UVec4D`](https://rerun.io/docs/reference/types/datatypes/uvec4d.md): A uint vector in 4D space.
* [`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md): A string of text, encoded as UTF-8.
* [`Utf8Pair`](https://rerun.io/docs/reference/types/datatypes/utf8pair.md): Stores a tuple of UTF-8 strings.
* [`Uuid`](https://rerun.io/docs/reference/types/datatypes/uuid.md): A 16-byte UUID.
* [`Vec2D`](https://rerun.io/docs/reference/types/datatypes/vec2d.md): A vector in 2D space.
* [`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md): A vector in 3D space.
* [`Vec4D`](https://rerun.io/docs/reference/types/datatypes/vec4d.md): A vector in 4D space.
* [`VideoTimestamp`](https://rerun.io/docs/reference/types/datatypes/video_timestamp.md): Presentation timestamp within a [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video).
* [`ViewCoordinates`](https://rerun.io/docs/reference/types/datatypes/view_coordinates.md): How we interpret the coordinate system of an entity/space.
* [`VisibleTimeRange`](https://rerun.io/docs/reference/types/datatypes/visible_time_range.md): Visible time range bounds for a specific timeline.
# Archetypes
Archetypes are bundles of components for which the Rerun viewer has first-class
built-in support. See [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md) and
[Visualizers and Overrides](https://rerun.io/docs/concepts/visualization/customize-views.md) for more information.
This page lists all built-in archetypes.
## Geospatial
* [`GeoLineStrings`](https://rerun.io/docs/reference/types/archetypes/geo_line_strings.md): Geospatial line strings with positions expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees), and optional colors and radii.
* [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md): Geospatial points with positions expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees), and optional colors and radii.
## Graph
* [`GraphEdges`](https://rerun.io/docs/reference/types/archetypes/graph_edges.md): A list of edges in a graph.
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md): A list of nodes in a graph with optional labels, colors, etc.
## Image & tensor
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md): A depth image, i.e. as captured by a depth camera.
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md): A depth image encoded with a codec (e.g. RVL or PNG).
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md): An image encoded as e.g. a JPEG or PNG.
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md): A monochrome or color image.
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md): An image made up of integer [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id)s.
* [`Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor.md): An N-dimensional array of numbers.
## MCAP
* [`McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel.md): A channel within an MCAP file that defines how messages are structured and encoded.
* [`McapMessage`](https://rerun.io/docs/reference/types/archetypes/mcap_message.md): The binary payload of a single MCAP message, without metadata.
* [`McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema.md): A schema definition that describes the structure of messages in an MCAP file.
* [`McapStatistics`](https://rerun.io/docs/reference/types/archetypes/mcap_statistics.md): Recording-level statistics about an MCAP file.
## Plotting
* [`BarChart`](https://rerun.io/docs/reference/types/archetypes/bar_chart.md): A bar chart.
* [`Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars.md): One or more double-precision scalar values, e.g. for use for time-series plots.
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md): Define the style properties for one or more line series in a chart.
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md): Define the style properties for one or more point series (scatter plot) in a chart.
* [`StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change.md): A state change, representing a transition of an entity into a new state.
* [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md): Define the style and mapping for state values in a state timeline view.
## Spatial 2D
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md): 2D arrows with optional colors, radii, labels, etc.
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md): 2D boxes with half-extents and optional center, colors etc.
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md): 2D ellipses with half-extents (semi-axes) and optional center, colors etc.
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md): 2D line strips with positions and optional colors, radii, labels, etc.
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md): A 2D point cloud with positions and optional colors, radii, labels, etc.
## Spatial 3D
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md): 3D arrows with optional colors, radii, labels, etc.
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md): A prepacked 3D asset (`.gltf`, `.glb`, `.obj`, `.stl`, etc.).
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md): 3D boxes with half-extents and optional center, rotations, colors etc.
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md): 3D capsules; cylinders with hemispherical caps.
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md): 3D cylinders with flat caps.
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md): 3D ellipsoids or spheres.
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md): A 2D grid map stored as raster data in an image buffer, with a cell size in scene units and pose.
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md): One or more transforms applied on the current entity's transform frame.
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md): 3D line strips with positions and optional colors, radii, labels, etc.
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md): A 3D triangle mesh as specified by its per-mesh and per-vertex properties.
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md): A 3D point cloud with positions and optional colors, radii, labels, etc.
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md): A sparse 3D voxel grid map with grid indices and voxel dimensions.
## Text
* [`TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document.md): A text element intended to be displayed in its own text box.
* [`TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log.md): A log entry in a text log, comprised of a text body and its log level.
## Transforms
* [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md): Specifies the coordinate frame for an entity.
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md): Camera perspective projection (a.k.a. intrinsics).
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md): A transform between two 3D spaces, i.e. a pose.
* [`TransformAxes3D`](https://rerun.io/docs/reference/types/archetypes/transform_axes3d.md): A visual representation of a [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
* [`ViewCoordinates`](https://rerun.io/docs/reference/types/archetypes/view_coordinates.md): How we interpret the coordinate system of an entity/space.
## Video
* [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md): A video binary.
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md): References a single video frame.
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md): Video stream consisting of raw video chunks.
## Other
* [`AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context.md): The annotation context provides additional information on how to display entities.
* [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md): Empties all the components of an entity.
* [`RecordingInfo`](https://rerun.io/docs/reference/types/archetypes/recording_info.md): A list of properties associated with a recording.
# Views
Views are the panels shown in the viewer's viewport and the primary means of inspecting & visualizing previously logged data. This page lists all built-in views.
* [`BarChartView`](https://rerun.io/docs/reference/types/views/bar_chart_view.md): A bar chart view.
* [`DataframeView`](https://rerun.io/docs/reference/types/views/dataframe_view.md): A view to display any data in a tabular form.
* [`GraphView`](https://rerun.io/docs/reference/types/views/graph_view.md): A graph view to display time-variying, directed or undirected graph visualization.
* [`MapView`](https://rerun.io/docs/reference/types/views/map_view.md): A 2D map view to display geospatial primitives.
* [`Spatial2DView`](https://rerun.io/docs/reference/types/views/spatial2d_view.md): For viewing spatial 2D data.
* [`Spatial3DView`](https://rerun.io/docs/reference/types/views/spatial3d_view.md): For viewing spatial 3D data.
* [`StateTimelineView`](https://rerun.io/docs/reference/types/views/state_timeline_view.md): A view for displaying state transitions over time, for use with [`archetypes.StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change).
* [`TensorView`](https://rerun.io/docs/reference/types/views/tensor_view.md): A view on a tensor of any dimensionality.
* [`TextDocumentView`](https://rerun.io/docs/reference/types/views/text_document_view.md): A view of a single text document, for use with [`archetypes.TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document).
* [`TextLogView`](https://rerun.io/docs/reference/types/views/text_log_view.md): A view of a text log, for use with [`archetypes.TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log).
* [`TimeSeriesView`](https://rerun.io/docs/reference/types/views/time_series_view.md): A time series view for scalars over time, for use with [`archetypes.Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars).
# Components
Components are the fundamental unit of logging in Rerun. This page lists all built-in components.
An entity can only ever contain a single array of any given component type.
If you log the same component several times on an entity, the last value (or array of values) will overwrite the previous.
For more information on the relationship between **archetypes** and **components**, check out the concept page
on [Entities and Components](https://rerun.io/docs/concepts/logging-and-ingestion/entity-component.md).
* [`AggregationPolicy`](https://rerun.io/docs/reference/types/components/aggregation_policy.md): Policy for aggregation of multiple scalar plot values.
* [`AlbedoFactor`](https://rerun.io/docs/reference/types/components/albedo_factor.md): A color multiplier, usually applied to a whole entity, e.g. a mesh.
* [`AnnotationContext`](https://rerun.io/docs/reference/types/components/annotation_context.md): The annotation context provides additional information on how to display entities.
* [`AxisLength`](https://rerun.io/docs/reference/types/components/axis_length.md): The length of an axis in local units of the space.
* [`Blob`](https://rerun.io/docs/reference/types/components/blob.md): A binary blob of data.
* [`CellSize`](https://rerun.io/docs/reference/types/components/cell_size.md): The metric size of one grid cell in local scene units.
* [`ChannelId`](https://rerun.io/docs/reference/types/components/channel_id.md): A 16-bit ID representing an MCAP channel.
* [`ChannelMessageCounts`](https://rerun.io/docs/reference/types/components/channel_message_counts.md): A mapping of channel IDs to their respective message counts.
* [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md): A 16-bit ID representing a type of semantic class.
* [`ClearIsRecursive`](https://rerun.io/docs/reference/types/components/clear_is_recursive.md): Configures how a clear operation should behave - recursive or not.
* [`Color`](https://rerun.io/docs/reference/types/components/color.md): An RGBA color with unmultiplied/separate alpha, in sRGB gamma space with linear alpha.
* [`Colormap`](https://rerun.io/docs/reference/types/components/colormap.md): Colormap for mapping scalar values within a given range to a color.
* [`Count`](https://rerun.io/docs/reference/types/components/count.md): A generic count value.
* [`DepthMeter`](https://rerun.io/docs/reference/types/components/depth_meter.md): The world->depth map scaling factor.
* [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md): Draw order of 2D elements. Higher values are drawn on top of lower values.
* [`EntityPath`](https://rerun.io/docs/reference/types/components/entity_path.md): A path to an entity, usually to reference some data that is part of the target entity.
* [`FillMode`](https://rerun.io/docs/reference/types/components/fill_mode.md): How a geometric shape is drawn and colored.
* [`FillRatio`](https://rerun.io/docs/reference/types/components/fill_ratio.md): How much a primitive fills out the available space.
* [`GammaCorrection`](https://rerun.io/docs/reference/types/components/gamma_correction.md): A gamma correction value to be used with a scalar value or color.
* [`GeoLineString`](https://rerun.io/docs/reference/types/components/geo_line_string.md): A geospatial line string expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees).
* [`GraphEdge`](https://rerun.io/docs/reference/types/components/graph_edge.md): An edge in a graph connecting two nodes.
* [`GraphNode`](https://rerun.io/docs/reference/types/components/graph_node.md): A string-based ID representing a node in a graph.
* [`GraphType`](https://rerun.io/docs/reference/types/components/graph_type.md): Specifies if a graph has directed or undirected edges.
* [`HalfSize2D`](https://rerun.io/docs/reference/types/components/half_size2d.md): Half-size (radius) of a 2D box.
* [`HalfSize3D`](https://rerun.io/docs/reference/types/components/half_size3d.md): Half-size (radius) of a 3D box.
* [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md): A buffer that is known to store image data.
* [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md): The metadata describing the contents of a [`components.ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer).
* [`ImagePlaneDistance`](https://rerun.io/docs/reference/types/components/image_plane_distance.md): The distance from the camera origin to the image plane when the projection is shown in a 3D viewer.
* [`Interactive`](https://rerun.io/docs/reference/types/components/interactive.md): Whether the entity can be interacted with.
* [`InterpolationMode`](https://rerun.io/docs/reference/types/components/interpolation_mode.md): Specifies how values between data points are interpolated in time series.
* [`IsKeyframe`](https://rerun.io/docs/reference/types/components/is_keyframe.md): Whether a [`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample) contains a keyframe (also known as a sync sample or IDR).
* [`KeyValuePairs`](https://rerun.io/docs/reference/types/components/key_value_pairs.md): A map of string keys to string values.
* [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md): A 16-bit ID representing a type of semantic keypoint within a class.
* [`LatLon`](https://rerun.io/docs/reference/types/components/lat_lon.md): A geospatial position expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees).
* [`Length`](https://rerun.io/docs/reference/types/components/length.md): Length, or one-dimensional size.
* [`LineStrip2D`](https://rerun.io/docs/reference/types/components/line_strip2d.md): A line strip in 2D space.
* [`LineStrip3D`](https://rerun.io/docs/reference/types/components/line_strip3d.md): A line strip in 3D space.
* [`LinearSpeed`](https://rerun.io/docs/reference/types/components/linear_speed.md): Linear speed, used for translation speed for example.
* [`MagnificationFilter`](https://rerun.io/docs/reference/types/components/magnification_filter.md): Filter used when a single texel/pixel of an image is displayed larger than a single screen pixel.
* [`MarkerShape`](https://rerun.io/docs/reference/types/components/marker_shape.md): The visual appearance of a point in e.g. a 2D plot.
* [`MarkerSize`](https://rerun.io/docs/reference/types/components/marker_size.md): Radius of a marker of a point in e.g. a 2D plot, measured in UI points.
* [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md): A standardized media type (RFC2046, formerly known as MIME types), encoded as a string.
* [`MeshFaceRendering`](https://rerun.io/docs/reference/types/components/mesh_face_rendering.md): Determines which faces of a mesh are rendered.
* [`Name`](https://rerun.io/docs/reference/types/components/name.md): A display name, typically for an entity or a item like a plot series.
* [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md): Degree of transparency ranging from 0.0 (fully transparent) to 1.0 (fully opaque).
* [`PinholeProjection`](https://rerun.io/docs/reference/types/components/pinhole_projection.md): Camera projection, from image coordinates to view coordinates.
* [`Plane3D`](https://rerun.io/docs/reference/types/components/plane3d.md): An infinite 3D plane represented by a unit normal vector and a distance.
* [`PointShading`](https://rerun.io/docs/reference/types/components/point_shading.md): Defines how points are shaded.
* [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md): A position in 2D space.
* [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md): A position in 3D space.
* [`Radius`](https://rerun.io/docs/reference/types/components/radius.md): The radius of something, e.g. a point.
* [`Range1D`](https://rerun.io/docs/reference/types/components/range1d.md): A 1D range, specifying a lower and upper bound.
* [`Resolution`](https://rerun.io/docs/reference/types/components/resolution.md): Pixel resolution width & height, e.g. of a camera sensor.
* [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md): 3D rotation represented by a rotation around a given axis.
* [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md): A 3D rotation expressed as a quaternion.
* [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md): A scalar value, encoded as a 64-bit floating point.
* [`Scale3D`](https://rerun.io/docs/reference/types/components/scale3d.md): A 3D scale factor.
* [`SchemaId`](https://rerun.io/docs/reference/types/components/schema_id.md): A 16-bit unique identifier for a schema within the MCAP file.
* [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md): Whether the entity's [`components.Text`](https://rerun.io/docs/reference/types/components/text) label is shown.
* [`StrokeWidth`](https://rerun.io/docs/reference/types/components/stroke_width.md): The width of a stroke specified in UI points.
* [`TensorData`](https://rerun.io/docs/reference/types/components/tensor_data.md): An N-dimensional array of numbers.
* [`TensorDimensionIndexSelection`](https://rerun.io/docs/reference/types/components/tensor_dimension_index_selection.md): Specifies a concrete index on a tensor dimension.
* [`TensorHeightDimension`](https://rerun.io/docs/reference/types/components/tensor_height_dimension.md): Specifies which dimension to use for height.
* [`TensorWidthDimension`](https://rerun.io/docs/reference/types/components/tensor_width_dimension.md): Specifies which dimension to use for width.
* [`Texcoord2D`](https://rerun.io/docs/reference/types/components/texcoord2d.md): A 2D texture UV coordinate.
* [`Text`](https://rerun.io/docs/reference/types/components/text.md): A string of text, e.g. for labels and text documents.
* [`TextLogLevel`](https://rerun.io/docs/reference/types/components/text_log_level.md): The severity level of a text log message.
* [`Timestamp`](https://rerun.io/docs/reference/types/components/timestamp.md): When the recording started.
* [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md): A string identifier for a transform frame.
* [`TransformMat3x3`](https://rerun.io/docs/reference/types/components/transform_mat3x3.md): A 3x3 transformation matrix Matrix.
* [`TransformRelation`](https://rerun.io/docs/reference/types/components/transform_relation.md): Specifies relation a spatial transform describes.
* [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md): A translation vector in 3D space.
* [`TriangleIndices`](https://rerun.io/docs/reference/types/components/triangle_indices.md): The three indices of a triangle in a triangle mesh.
* [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md): Range of expected or valid values, specifying a lower and upper bound.
* [`Vector2D`](https://rerun.io/docs/reference/types/components/vector2d.md): A vector in 2D space.
* [`Vector3D`](https://rerun.io/docs/reference/types/components/vector3d.md): A vector in 3D space.
* [`VideoCodec`](https://rerun.io/docs/reference/types/components/video_codec.md): The codec used to encode video stored in [`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample).
* [`VideoSample`](https://rerun.io/docs/reference/types/components/video_sample.md): Video sample data (also known as "video chunk").
* [`VideoTimestamp`](https://rerun.io/docs/reference/types/components/video_timestamp.md): Timestamp inside a [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video).
* [`ViewCoordinates`](https://rerun.io/docs/reference/types/components/view_coordinates.md): How we interpret the coordinate system of an entity/space.
* [`Visible`](https://rerun.io/docs/reference/types/components/visible.md): Whether the container, view, entity or instance is currently visible.
* [`VoxelIndex`](https://rerun.io/docs/reference/types/components/voxel_index.md): Integer index of a voxel in a sparse 3D voxel grid.
* [`VoxelSize`](https://rerun.io/docs/reference/types/components/voxel_size.md): The scene-unit dimensions of one voxel in a sparse 3D voxel grid.
* [`VoxelValue`](https://rerun.io/docs/reference/types/components/voxel_value.md): Optional scalar occupancy or value associated with a voxel.
# PinholeProjection
Camera projection, from image coordinates to view coordinates.
Child from parent.
Image coordinates from camera view coordinates.
Example:
```text
1496.1 0.0 980.5
0.0 1496.1 744.5
0.0 0.0 1.0
```
## Rerun datatype
[`Mat3x3`](https://rerun.io/docs/reference/types/datatypes/mat3x3.md)
## Arrow datatype
```
FixedSizeList(9 x non-null Float32)
```
## API reference links
* π [C++ API docs for `PinholeProjection`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1PinholeProjection.html)
* π [Python API docs for `PinholeProjection`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.PinholeProjection)
* π¦ [Rust API docs for `PinholeProjection`](https://docs.rs/rerun/latest/rerun/components/struct.PinholeProjection.html)
## Used by
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
# HalfSize3D
Half-size (radius) of a 3D box.
Measured in its local coordinate system.
The box extends both in negative and positive direction along each axis.
Negative sizes indicate that the box is flipped along the respective axis, but this has no effect on how it is displayed.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `HalfSize3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1HalfSize3D.html)
* π [Python API docs for `HalfSize3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.HalfSize3D)
* π¦ [Rust API docs for `HalfSize3D`](https://docs.rs/rerun/latest/rerun/components/struct.HalfSize3D.html)
## Used by
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
# PointShading
Defines how points are shaded.
## Variants
#### `Gradient` = 1
Radial gradient for a spherical shadow effect.
#### `Flat` = 2
Flat shading.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `PointShading`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `PointShading`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.PointShading)
* π¦ [Rust API docs for `PointShading`](https://docs.rs/rerun/latest/rerun/components/enum.PointShading.html)
## Used by
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
# GraphEdge
An edge in a graph connecting two nodes.
## Rerun datatype
[`Utf8Pair`](https://rerun.io/docs/reference/types/datatypes/utf8pair.md)
## Arrow datatype
```
Struct(
"first": non-null Utf8
"second": non-null Utf8
)
```
## API reference links
* π [C++ API docs for `GraphEdge`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1GraphEdge.html)
* π [Python API docs for `GraphEdge`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.GraphEdge)
* π¦ [Rust API docs for `GraphEdge`](https://docs.rs/rerun/latest/rerun/components/struct.GraphEdge.html)
## Used by
* [`GraphEdges`](https://rerun.io/docs/reference/types/archetypes/graph_edges.md)
# SchemaId
A 16-bit unique identifier for a schema within the MCAP file.
## Rerun datatype
[`UInt16`](https://rerun.io/docs/reference/types/datatypes/uint16.md)
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `SchemaId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1SchemaId.html)
* π [Python API docs for `SchemaId`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.SchemaId)
* π¦ [Rust API docs for `SchemaId`](https://docs.rs/rerun/latest/rerun/components/struct.SchemaId.html)
## Used by
* [`McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema.md)
# KeypointId
A 16-bit ID representing a type of semantic keypoint within a class.
## Rerun datatype
[`KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id.md)
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `KeypointId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1KeypointId.html)
* π [Python API docs for `KeypointId`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.KeypointId)
* π¦ [Rust API docs for `KeypointId`](https://docs.rs/rerun/latest/rerun/components/struct.KeypointId.html)
## Used by
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
# Name
A display name, typically for an entity or a item like a plot series.
## Rerun datatype
[`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `Name`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Name.html)
* π [Python API docs for `Name`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Name)
* π¦ [Rust API docs for `Name`](https://docs.rs/rerun/latest/rerun/components/struct.Name.html)
## Used by
* [`RecordingInfo`](https://rerun.io/docs/reference/types/archetypes/recording_info.md)
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
# Range1D
A 1D range, specifying a lower and upper bound.
## Rerun datatype
[`Range1D`](https://rerun.io/docs/reference/types/datatypes/range1d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float64)
```
## API reference links
* π [C++ API docs for `Range1D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Range1D.html)
* π [Python API docs for `Range1D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Range1D)
* π¦ [Rust API docs for `Range1D`](https://docs.rs/rerun/latest/rerun/components/struct.Range1D.html)
# ImageBuffer
A buffer that is known to store image data.
To interpret the contents of this buffer, see, [`components.ImageFormat`](https://rerun.io/docs/reference/types/components/image_format).
## Rerun datatype
[`Blob`](https://rerun.io/docs/reference/types/datatypes/blob.md)
## Arrow datatype
```
List(non-null UInt8)
```
## API reference links
* π [C++ API docs for `ImageBuffer`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ImageBuffer.html)
* π [Python API docs for `ImageBuffer`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ImageBuffer)
* π¦ [Rust API docs for `ImageBuffer`](https://docs.rs/rerun/latest/rerun/components/struct.ImageBuffer.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md)
# ChannelId
A 16-bit ID representing an MCAP channel.
Used to identify specific channels within an MCAP file.
## Rerun datatype
[`UInt16`](https://rerun.io/docs/reference/types/datatypes/uint16.md)
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `ChannelId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ChannelId.html)
* π [Python API docs for `ChannelId`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ChannelId)
* π¦ [Rust API docs for `ChannelId`](https://docs.rs/rerun/latest/rerun/components/struct.ChannelId.html)
## Used by
* [`McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel.md)
# LineStrip3D
A line strip in 3D space.
A line strip is a list of points connected by line segments. It can be used to draw
approximations of smooth curves.
The points will be connected in order, like so:
```text
2------3 5
/ \ /
0----1 \ /
4
```
## Arrow datatype
```
List(non-null FixedSizeList(3 x non-null Float32))
```
## API reference links
* π [C++ API docs for `LineStrip3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1LineStrip3D.html)
* π [Python API docs for `LineStrip3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.LineStrip3D)
* π¦ [Rust API docs for `LineStrip3D`](https://docs.rs/rerun/latest/rerun/components/struct.LineStrip3D.html)
## Used by
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md)
# Scalar
A scalar value, encoded as a 64-bit floating point.
Used for time series plots.
## Rerun datatype
[`Float64`](https://rerun.io/docs/reference/types/datatypes/float64.md)
## Arrow datatype
```
Float64
```
## API reference links
* π [C++ API docs for `Scalar`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Scalar.html)
* π [Python API docs for `Scalar`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Scalar)
* π¦ [Rust API docs for `Scalar`](https://docs.rs/rerun/latest/rerun/components/struct.Scalar.html)
## Used by
* [`Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars.md)
# DrawOrder
Draw order of 2D elements. Higher values are drawn on top of lower values.
An entity can have only a single draw order component.
Within an entity draw order is governed by the order of the components.
Draw order for entities with the same draw order is generally undefined.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `DrawOrder`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1DrawOrder.html)
* π [Python API docs for `DrawOrder`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.DrawOrder)
* π¦ [Rust API docs for `DrawOrder`](https://docs.rs/rerun/latest/rerun/components/struct.DrawOrder.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md)
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md)
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
# GraphType
Specifies if a graph has directed or undirected edges.
## Variants
#### `Undirected` = 1
The graph has undirected edges.
#### `Directed` = 2
The graph has directed edges.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `GraphType`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `GraphType`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.GraphType)
* π¦ [Rust API docs for `GraphType`](https://docs.rs/rerun/latest/rerun/components/enum.GraphType.html)
## Used by
* [`GraphEdges`](https://rerun.io/docs/reference/types/archetypes/graph_edges.md)
# MarkerShape
The visual appearance of a point in e.g. a 2D plot.
## Variants
#### `Circle` = 1
`βΊ`
#### `Diamond` = 2
`β`
#### `Square` = 3
`βΌοΈ`
#### `Cross` = 4
`x`
#### `Plus` = 5
`+`
#### `Up` = 6
`β²`
#### `Down` = 7
`βΌ`
#### `Left` = 8
`β`
#### `Right` = 9
`βΆ`
#### `Asterisk` = 10
`*`
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `MarkerShape`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `MarkerShape`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.MarkerShape)
* π¦ [Rust API docs for `MarkerShape`](https://docs.rs/rerun/latest/rerun/components/enum.MarkerShape.html)
## Used by
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
# MediaType
A standardized media type (RFC2046, formerly known as MIME types), encoded as a string.
The complete reference of officially registered media types is maintained by the IANA and can be
consulted at .
## Rerun datatype
[`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `MediaType`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1MediaType.html)
* π [Python API docs for `MediaType`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.MediaType)
* π¦ [Rust API docs for `MediaType`](https://docs.rs/rerun/latest/rerun/components/struct.MediaType.html)
## Used by
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md)
* [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document.md)
# VideoTimestamp
Timestamp inside a [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video).
## Rerun datatype
[`VideoTimestamp`](https://rerun.io/docs/reference/types/datatypes/video_timestamp.md)
## Arrow datatype
```
Int64
```
## API reference links
* π [C++ API docs for `VideoTimestamp`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1VideoTimestamp.html)
* π [Python API docs for `VideoTimestamp`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VideoTimestamp)
* π¦ [Rust API docs for `VideoTimestamp`](https://docs.rs/rerun/latest/rerun/components/struct.VideoTimestamp.html)
## Used by
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md)
# TransformMat3x3
A 3x3 transformation matrix Matrix.
3x3 matrixes are able to represent any affine transformation in 3D space,
i.e. rotation, scaling, shearing, reflection etc.
Matrices in Rerun are stored as flat list of coefficients in column-major order:
```text
column 0 column 1 column 2
-------------------------------------------------
row 0 | flat_columns[0] flat_columns[3] flat_columns[6]
row 1 | flat_columns[1] flat_columns[4] flat_columns[7]
row 2 | flat_columns[2] flat_columns[5] flat_columns[8]
```
## Rerun datatype
[`Mat3x3`](https://rerun.io/docs/reference/types/datatypes/mat3x3.md)
## Arrow datatype
```
FixedSizeList(9 x non-null Float32)
```
## API reference links
* π [C++ API docs for `TransformMat3x3`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TransformMat3x3.html)
* π [Python API docs for `TransformMat3x3`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TransformMat3x3)
* π¦ [Rust API docs for `TransformMat3x3`](https://docs.rs/rerun/latest/rerun/components/struct.TransformMat3x3.html)
## Used by
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
# TensorWidthDimension
Specifies which dimension to use for width.
## Rerun datatype
[`TensorDimensionSelection`](https://rerun.io/docs/reference/types/datatypes/tensor_dimension_selection.md)
## Arrow datatype
```
Struct(
"dimension": non-null UInt32
"invert": non-null Boolean
)
```
## API reference links
* π [C++ API docs for `TensorWidthDimension`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TensorWidthDimension.html)
* π [Python API docs for `TensorWidthDimension`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TensorWidthDimension)
* π¦ [Rust API docs for `TensorWidthDimension`](https://docs.rs/rerun/latest/rerun/components/struct.TensorWidthDimension.html)
# VoxelValue
Optional scalar occupancy or value associated with a voxel.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `VoxelValue`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1VoxelValue.html)
* π [Python API docs for `VoxelValue`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VoxelValue)
* π¦ [Rust API docs for `VoxelValue`](https://docs.rs/rerun/latest/rerun/components/struct.VoxelValue.html)
## Used by
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# Scale3D
A 3D scale factor.
A scale of 1.0 means no scaling.
A scale of 2.0 means doubling the size.
Each component scales along the corresponding axis.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Scale3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Scale3D.html)
* π [Python API docs for `Scale3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Scale3D)
* π¦ [Rust API docs for `Scale3D`](https://docs.rs/rerun/latest/rerun/components/struct.Scale3D.html)
## Used by
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
# Vector2D
A vector in 2D space.
## Rerun datatype
[`Vec2D`](https://rerun.io/docs/reference/types/datatypes/vec2d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Vector2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Vector2D.html)
* π [Python API docs for `Vector2D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Vector2D)
* π¦ [Rust API docs for `Vector2D`](https://docs.rs/rerun/latest/rerun/components/struct.Vector2D.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
# ImageFormat
The metadata describing the contents of a [`components.ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer).
## Rerun datatype
[`ImageFormat`](https://rerun.io/docs/reference/types/datatypes/image_format.md)
## Arrow datatype
```
Struct(
"width": non-null UInt32
"height": non-null UInt32
"pixel_format": UInt8
"color_model": UInt8
"channel_datatype": UInt8
)
```
## API reference links
* π [C++ API docs for `ImageFormat`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ImageFormat.html)
* π [Python API docs for `ImageFormat`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ImageFormat)
* π¦ [Rust API docs for `ImageFormat`](https://docs.rs/rerun/latest/rerun/components/struct.ImageFormat.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md)
# Text
A string of text, e.g. for labels and text documents.
## Rerun datatype
[`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `Text`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Text.html)
* π [Python API docs for `Text`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Text)
* π¦ [Rust API docs for `Text`](https://docs.rs/rerun/latest/rerun/components/struct.Text.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md)
* [`McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel.md)
* [`McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
* [`StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change.md)
* [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md)
* [`TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document.md)
* [`TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log.md)
# Position2D
A position in 2D space.
## Rerun datatype
[`Vec2D`](https://rerun.io/docs/reference/types/datatypes/vec2d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Position2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Position2D.html)
* π [Python API docs for `Position2D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Position2D)
* π¦ [Rust API docs for `Position2D`](https://docs.rs/rerun/latest/rerun/components/struct.Position2D.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
# AlbedoFactor
A color multiplier, usually applied to a whole entity, e.g. a mesh.
## Rerun datatype
[`Rgba32`](https://rerun.io/docs/reference/types/datatypes/rgba32.md)
## Arrow datatype
```
UInt32
```
## API reference links
* π [C++ API docs for `AlbedoFactor`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1AlbedoFactor.html)
* π [Python API docs for `AlbedoFactor`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.AlbedoFactor)
* π¦ [Rust API docs for `AlbedoFactor`](https://docs.rs/rerun/latest/rerun/components/struct.AlbedoFactor.html)
## Used by
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
# AggregationPolicy
Policy for aggregation of multiple scalar plot values.
This is used for lines in plots when the X axis distance of individual points goes below a single pixel,
i.e. a single pixel covers more than one tick worth of data. It can greatly improve performance
(and readability) in such situations as it prevents overdraw.
## Variants
#### `Off` = 1
No aggregation.
#### `Average` = 2
Average all points in the range together.
#### `Max` = 3
Keep only the maximum values in the range.
#### `Min` = 4
Keep only the minimum values in the range.
#### `MinMax` = 5
Keep both the minimum and maximum values in the range.
This will yield two aggregated points instead of one, effectively creating a vertical line.
#### `MinMaxAverage` = 6
Find both the minimum and maximum values in the range, then use the average of those.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `AggregationPolicy`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `AggregationPolicy`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.AggregationPolicy)
* π¦ [Rust API docs for `AggregationPolicy`](https://docs.rs/rerun/latest/rerun/components/enum.AggregationPolicy.html)
## Used by
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
# StrokeWidth
The width of a stroke specified in UI points.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `StrokeWidth`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1StrokeWidth.html)
* π [Python API docs for `StrokeWidth`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.StrokeWidth)
* π¦ [Rust API docs for `StrokeWidth`](https://docs.rs/rerun/latest/rerun/components/struct.StrokeWidth.html)
## Used by
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
# Resolution
Pixel resolution width & height, e.g. of a camera sensor.
Typically in integer units, but for some use cases floating point may be used.
## Rerun datatype
[`Vec2D`](https://rerun.io/docs/reference/types/datatypes/vec2d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Resolution`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Resolution.html)
* π [Python API docs for `Resolution`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Resolution)
* π¦ [Rust API docs for `Resolution`](https://docs.rs/rerun/latest/rerun/components/struct.Resolution.html)
## Used by
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
# Visible
Whether the container, view, entity or instance is currently visible.
## Rerun datatype
[`Bool`](https://rerun.io/docs/reference/types/datatypes/bool.md)
## Arrow datatype
```
Boolean
```
## API reference links
* π [C++ API docs for `Visible`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Visible.html)
* π [Python API docs for `Visible`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Visible)
* π¦ [Rust API docs for `Visible`](https://docs.rs/rerun/latest/rerun/components/struct.Visible.html)
## Used by
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
* [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md)
# LatLon
A geospatial position expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees).
## Rerun datatype
[`DVec2D`](https://rerun.io/docs/reference/types/datatypes/dvec2d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float64)
```
## API reference links
* π [C++ API docs for `LatLon`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1LatLon.html)
* π [Python API docs for `LatLon`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.LatLon)
* π¦ [Rust API docs for `LatLon`](https://docs.rs/rerun/latest/rerun/components/struct.LatLon.html)
## Used by
* [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md)
# Plane3D
An infinite 3D plane represented by a unit normal vector and a distance.
Any point P on the plane fulfills the equation `dot(xyz, P) - d = 0`,
where `xyz` is the plane's normal and `d` the distance of the plane from the origin.
This representation is also known as the Hesse normal form.
Note: although the normal will be passed through to the
datastore as provided, when used in the Viewer, planes will always be normalized.
I.e. the plane with xyz = (2, 0, 0), d = 1 is equivalent to xyz = (1, 0, 0), d = 0.5
## Rerun datatype
[`Plane3D`](https://rerun.io/docs/reference/types/datatypes/plane3d.md)
## Arrow datatype
```
FixedSizeList(4 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Plane3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Plane3D.html)
* π [Python API docs for `Plane3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Plane3D)
* π¦ [Rust API docs for `Plane3D`](https://docs.rs/rerun/latest/rerun/components/struct.Plane3D.html)
# Vector3D
A vector in 3D space.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Vector3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Vector3D.html)
* π [Python API docs for `Vector3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Vector3D)
* π¦ [Rust API docs for `Vector3D`](https://docs.rs/rerun/latest/rerun/components/struct.Vector3D.html)
## Used by
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
# Timestamp
When the recording started.
Should be an absolute time, i.e. relative to Unix Epoch.
## Rerun datatype
[`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md)
## Arrow datatype
```
Int64
```
## API reference links
* π [C++ API docs for `Timestamp`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Timestamp.html)
* π [Python API docs for `Timestamp`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Timestamp)
* π¦ [Rust API docs for `Timestamp`](https://docs.rs/rerun/latest/rerun/components/struct.Timestamp.html)
## Used by
* [`McapStatistics`](https://rerun.io/docs/reference/types/archetypes/mcap_statistics.md)
* [`RecordingInfo`](https://rerun.io/docs/reference/types/archetypes/recording_info.md)
# RotationQuat
A 3D rotation expressed as a quaternion.
Note: although the x,y,z,w components of the quaternion will be passed through to the
datastore as provided, when used in the Viewer, quaternions will always be normalized.
If normalization fails the rotation is treated as an invalid transform.
## Rerun datatype
[`Quaternion`](https://rerun.io/docs/reference/types/datatypes/quaternion.md)
## Arrow datatype
```
FixedSizeList(4 x non-null Float32)
```
## API reference links
* π [C++ API docs for `RotationQuat`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1RotationQuat.html)
* π [Python API docs for `RotationQuat`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.RotationQuat)
* π¦ [Rust API docs for `RotationQuat`](https://docs.rs/rerun/latest/rerun/components/struct.RotationQuat.html)
## Used by
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# FillMode
How a geometric shape is drawn and colored.
## Variants
#### `MajorWireframe` = 1
Lines are drawn around the parts of the shape which directly correspond to the logged data.
Examples of what this means:
* An [`archetypes.Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d) will draw three axis-aligned ellipses that are cross-sections
of each ellipsoid, each of which displays two out of three of the sizes of the ellipsoid.
* For [`archetypes.Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d), it is the edges of the box, identical to [`components.FillMode#DenseWireframe`](https://rerun.io/docs/reference/types/components/fill_mode).
#### `DenseWireframe` = 2
Many lines are drawn to represent the surface of the shape in a see-through fashion.
Examples of what this means:
* An [`archetypes.Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d) will draw a wireframe triangle mesh that approximates each
ellipsoid.
* For [`archetypes.Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d), it is the edges of the box, identical to [`components.FillMode#MajorWireframe`](https://rerun.io/docs/reference/types/components/fill_mode).
#### `Solid` = 3
The surface of the shape is filled in with a solid color. No lines are drawn.
#### `TransparentFillMajorWireframe` = 4
The surface of the shape is filled in with a transparent color, with major wireframe lines on top.
This gives a good default appearance that shows both the shape's surface and its structure.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `FillMode`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `FillMode`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.FillMode)
* π¦ [Rust API docs for `FillMode`](https://docs.rs/rerun/latest/rerun/components/enum.FillMode.html)
## Used by
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
# ChannelMessageCounts
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A mapping of channel IDs to their respective message counts.
Used in MCAP statistics to track how many messages were recorded per channel.
## Arrow datatype
```
List(non-null Struct(
"channel_id": non-null UInt16
"message_count": non-null UInt64
))
```
## API reference links
* π [C++ API docs for `ChannelMessageCounts`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ChannelMessageCounts.html)
* π [Python API docs for `ChannelMessageCounts`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ChannelMessageCounts)
* π¦ [Rust API docs for `ChannelMessageCounts`](https://docs.rs/rerun/latest/rerun/components/struct.ChannelMessageCounts.html)
## Used by
* [`McapStatistics`](https://rerun.io/docs/reference/types/archetypes/mcap_statistics.md)
# LineStrip2D
A line strip in 2D space.
A line strip is a list of points connected by line segments. It can be used to draw
approximations of smooth curves.
The points will be connected in order, like so:
```text
2------3 5
/ \ /
0----1 \ /
4
```
## Arrow datatype
```
List(non-null FixedSizeList(2 x non-null Float32))
```
## API reference links
* π [C++ API docs for `LineStrip2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1LineStrip2D.html)
* π [Python API docs for `LineStrip2D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.LineStrip2D)
* π¦ [Rust API docs for `LineStrip2D`](https://docs.rs/rerun/latest/rerun/components/struct.LineStrip2D.html)
## Used by
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
# MeshFaceRendering
Determines which faces of a mesh are rendered.
For this purpose, we assume that the winding order of vertices in a mesh is
consistent and that front faces are defined as those with vertices in counter clockwise order.
## Variants
#### `DoubleSided` = 1
Show both back and front faces.
#### `Front` = 2
Only front faces are shown.
Front faces are assumed to have a counter clockwise vertex winding order on screen.
#### `Back` = 3
Only back faces are shown.
Back faces are assumed to have a clockwise vertex winding order on screen.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `MeshFaceRendering`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `MeshFaceRendering`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.MeshFaceRendering)
* π¦ [Rust API docs for `MeshFaceRendering`](https://docs.rs/rerun/latest/rerun/components/enum.MeshFaceRendering.html)
## Used by
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
# TensorDimensionIndexSelection
Specifies a concrete index on a tensor dimension.
## Rerun datatype
[`TensorDimensionIndexSelection`](https://rerun.io/docs/reference/types/datatypes/tensor_dimension_index_selection.md)
## Arrow datatype
```
Struct(
"dimension": non-null UInt32
"index": non-null UInt64
)
```
## API reference links
* π [C++ API docs for `TensorDimensionIndexSelection`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TensorDimensionIndexSelection.html)
* π [Python API docs for `TensorDimensionIndexSelection`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TensorDimensionIndexSelection)
* π¦ [Rust API docs for `TensorDimensionIndexSelection`](https://docs.rs/rerun/latest/rerun/components/struct.TensorDimensionIndexSelection.html)
# Color
An RGBA color with unmultiplied/separate alpha, in sRGB gamma space with linear alpha.
The color is stored as a 32-bit integer, where the most significant
byte is `R` and the least significant byte is `A`.
## Rerun datatype
[`Rgba32`](https://rerun.io/docs/reference/types/datatypes/rgba32.md)
## Arrow datatype
```
UInt32
```
## API reference links
* π [C++ API docs for `Color`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Color.html)
* π [Python API docs for `Color`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Color)
* π¦ [Rust API docs for `Color`](https://docs.rs/rerun/latest/rerun/components/struct.Color.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`BarChart`](https://rerun.io/docs/reference/types/archetypes/bar_chart.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GeoLineStrings`](https://rerun.io/docs/reference/types/archetypes/geo_line_strings.md)
* [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md)
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
* [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md)
* [`TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# AnnotationContext
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
The annotation context provides additional information on how to display entities.
Entities can use [`datatypes.ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id)s and [`datatypes.KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id)s to provide annotations, and
the labels and colors will be looked up in the appropriate
annotation context. We use the *first* annotation context we find in the
path-hierarchy when searching up through the ancestors of a given entity
path.
## Arrow datatype
```
List(non-null Struct(
"class_id": non-null UInt16
"class_description": non-null Struct(
"info": non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
)
"keypoint_annotations": non-null List(non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
))
"keypoint_connections": non-null List(non-null Struct(
"keypoint0": non-null UInt16
"keypoint1": non-null UInt16
))
)
))
```
## API reference links
* π [C++ API docs for `AnnotationContext`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1AnnotationContext.html)
* π [Python API docs for `AnnotationContext`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.AnnotationContext)
* π¦ [Rust API docs for `AnnotationContext`](https://docs.rs/rerun/latest/rerun/components/struct.AnnotationContext.html)
## Used by
* [`AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context.md)
# GraphNode
A string-based ID representing a node in a graph.
## Rerun datatype
[`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `GraphNode`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1GraphNode.html)
* π [Python API docs for `GraphNode`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.GraphNode)
* π¦ [Rust API docs for `GraphNode`](https://docs.rs/rerun/latest/rerun/components/struct.GraphNode.html)
## Used by
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
# TransformRelation
Specifies relation a spatial transform describes.
## Variants
#### `ParentFromChild` = 1
The transform describes how to transform into the parent entity's space.
E.g. a translation of (0, 1, 0) with this [`components.TransformRelation`](https://rerun.io/docs/reference/types/components/transform_relation) logged at `parent/child` means
that from the point of view of `parent`, `parent/child` is translated 1 unit along `parent`'s Y axis.
From perspective of `parent/child`, the `parent` entity is translated -1 unit along `parent/child`'s Y axis.
#### `ChildFromParent` = 2
The transform describes how to transform into the child entity's space.
E.g. a translation of (0, 1, 0) with this [`components.TransformRelation`](https://rerun.io/docs/reference/types/components/transform_relation) logged at `parent/child` means
that from the point of view of `parent`, `parent/child` is translated -1 unit along `parent`'s Y axis.
From perspective of `parent/child`, the `parent` entity is translated 1 unit along `parent/child`'s Y axis.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `TransformRelation`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `TransformRelation`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TransformRelation)
* π¦ [Rust API docs for `TransformRelation`](https://docs.rs/rerun/latest/rerun/components/enum.TransformRelation.html)
## Used by
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
# FillRatio
How much a primitive fills out the available space.
Used for instance to scale the points of the point cloud created from [`archetypes.DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image) projection in 3D views.
Valid range is from 0 to max float although typically values above 1.0 are not useful.
Defaults to 1.0.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `FillRatio`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1FillRatio.html)
* π [Python API docs for `FillRatio`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.FillRatio)
* π¦ [Rust API docs for `FillRatio`](https://docs.rs/rerun/latest/rerun/components/struct.FillRatio.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
# LinearSpeed
Linear speed, used for translation speed for example.
## Rerun datatype
[`Float64`](https://rerun.io/docs/reference/types/datatypes/float64.md)
## Arrow datatype
```
Float64
```
## API reference links
* π [C++ API docs for `LinearSpeed`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1LinearSpeed.html)
* π [Python API docs for `LinearSpeed`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.LinearSpeed)
* π¦ [Rust API docs for `LinearSpeed`](https://docs.rs/rerun/latest/rerun/components/struct.LinearSpeed.html)
# KeyValuePairs
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A map of string keys to string values.
This component can be used to attach arbitrary metadata or annotations to entities.
Each key-value pair is stored as a UTF-8 string mapping.
## Arrow datatype
```
List(non-null Struct(
"first": non-null Utf8
"second": non-null Utf8
))
```
## API reference links
* π [C++ API docs for `KeyValuePairs`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1KeyValuePairs.html)
* π [Python API docs for `KeyValuePairs`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.KeyValuePairs)
* π¦ [Rust API docs for `KeyValuePairs`](https://docs.rs/rerun/latest/rerun/components/struct.KeyValuePairs.html)
## Used by
* [`McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel.md)
# InterpolationMode
Specifies how values between data points are interpolated in time series.
## Variants
#### `Linear` = 1
Connect data points with straight line segments.
#### `StepAfter` = 2
Hold the previous value until the next data point, then jump.
The step occurs at the end of the interval.
#### `StepBefore` = 3
Jump to the new value immediately, then hold until the next data point.
The step occurs at the beginning of the interval.
#### `StepMid` = 4
Hold the previous value until the midpoint between data points, then jump to the new value.
The step occurs at the midpoint of the interval.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `InterpolationMode`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `InterpolationMode`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.InterpolationMode)
* π¦ [Rust API docs for `InterpolationMode`](https://docs.rs/rerun/latest/rerun/components/enum.InterpolationMode.html)
## Used by
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
# GeoLineString
A geospatial line string expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees).
## Arrow datatype
```
List(non-null FixedSizeList(2 x non-null Float64))
```
## API reference links
* π [C++ API docs for `GeoLineString`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1GeoLineString.html)
* π [Python API docs for `GeoLineString`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.GeoLineString)
* π¦ [Rust API docs for `GeoLineString`](https://docs.rs/rerun/latest/rerun/components/struct.GeoLineString.html)
## Used by
* [`GeoLineStrings`](https://rerun.io/docs/reference/types/archetypes/geo_line_strings.md)
# MarkerSize
Radius of a marker of a point in e.g. a 2D plot, measured in UI points.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `MarkerSize`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1MarkerSize.html)
* π [Python API docs for `MarkerSize`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.MarkerSize)
* π¦ [Rust API docs for `MarkerSize`](https://docs.rs/rerun/latest/rerun/components/struct.MarkerSize.html)
## Used by
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
# EntityPath
A path to an entity, usually to reference some data that is part of the target entity.
## Rerun datatype
[`EntityPath`](https://rerun.io/docs/reference/types/datatypes/entity_path.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `EntityPath`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1EntityPath.html)
* π [Python API docs for `EntityPath`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.EntityPath)
* π¦ [Rust API docs for `EntityPath`](https://docs.rs/rerun/latest/rerun/components/struct.EntityPath.html)
## Used by
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md)
# TensorHeightDimension
Specifies which dimension to use for height.
## Rerun datatype
[`TensorDimensionSelection`](https://rerun.io/docs/reference/types/datatypes/tensor_dimension_selection.md)
## Arrow datatype
```
Struct(
"dimension": non-null UInt32
"invert": non-null Boolean
)
```
## API reference links
* π [C++ API docs for `TensorHeightDimension`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TensorHeightDimension.html)
* π [Python API docs for `TensorHeightDimension`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TensorHeightDimension)
* π¦ [Rust API docs for `TensorHeightDimension`](https://docs.rs/rerun/latest/rerun/components/struct.TensorHeightDimension.html)
# DepthMeter
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
The world->depth map scaling factor.
This measures how many depth map units are in a world unit.
For instance, if a depth map uses millimeters and the world uses meters,
this value would be `1000`.
Note that the only effect on 2D views is the physical depth values shown when hovering the image.
In 3D views on the other hand, this affects where the points of the point cloud are placed.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `DepthMeter`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1DepthMeter.html)
* π [Python API docs for `DepthMeter`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.DepthMeter)
* π¦ [Rust API docs for `DepthMeter`](https://docs.rs/rerun/latest/rerun/components/struct.DepthMeter.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
# Position3D
A position in 3D space.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Position3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Position3D.html)
* π [Python API docs for `Position3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Position3D)
* π¦ [Rust API docs for `Position3D`](https://docs.rs/rerun/latest/rerun/components/struct.Position3D.html)
## Used by
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
# TriangleIndices
The three indices of a triangle in a triangle mesh.
## Rerun datatype
[`UVec3D`](https://rerun.io/docs/reference/types/datatypes/uvec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null UInt32)
```
## API reference links
* π [C++ API docs for `TriangleIndices`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TriangleIndices.html)
* π [Python API docs for `TriangleIndices`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TriangleIndices)
* π¦ [Rust API docs for `TriangleIndices`](https://docs.rs/rerun/latest/rerun/components/struct.TriangleIndices.html)
## Used by
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
# Colormap
Colormap for mapping scalar values within a given range to a color.
This provides a number of popular pre-defined colormaps.
In the future, the Rerun Viewer will allow users to define their own colormaps,
but currently the Viewer is limited to the types defined here.
## Variants
#### `Grayscale` = 1
A simple black to white gradient.
This is a sRGB gray gradient which is perceptually uniform.
#### `Inferno` = 2
The Inferno colormap from Matplotlib.
This is a perceptually uniform colormap.
It interpolates from black to red to bright yellow.
#### `Magma` = 3
The Magma colormap from Matplotlib.
This is a perceptually uniform colormap.
It interpolates from black to purple to white.
#### `Plasma` = 4
The Plasma colormap from Matplotlib.
This is a perceptually uniform colormap.
It interpolates from dark blue to purple to yellow.
#### `Turbo` = 5
Google's Turbo colormap map.
This is a perceptually non-uniform rainbow colormap addressing many issues of
more traditional rainbow colormaps like Jet.
It is more perceptually uniform without sharp transitions and is more colorblind-friendly.
Details:
[`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample)s using this codec should be formatted according the "Low overhead bitstream format",
as specified in Section 5.2 of the [AV1 specification](https://aomediacodec.github.io/av1-spec/#low-overhead-bitstream-format).
Each sample should be formatted as a sequence of OBUs (Open Bitstream Units) long enough to decode at least one video frame.
Samples containing keyframes must include a sequence header OBU before the `KEY_FRAME` OBU to enable
extraction of frame dimensions, bit depth, and color information. `INTRA_ONLY` frames are not treated
as keyframes since they may reference existing decoder state.
Enum value is the fourcc for 'av01' (the WebCodec string assigned to this codec) in big endian.
#### `H264` = 0x61766331
Advanced Video Coding (AVC/H.264)
See
[`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample)s using this codec should be formatted according to Annex B specification.
(Note that this is different from AVCC format found in MP4 files.
To learn more about Annex B, check for instance )
Key frames (IDR) require inclusion of a SPS (Sequence Parameter Set)
Enum value is the fourcc for 'avc1' (the WebCodec string assigned to this codec) in big endian.
#### `H265` = 0x68657631
High Efficiency Video Coding (HEVC/H.265)
See
[`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample)s using this codec should be formatted according to Annex B specification.
(Note that this is different from AVCC format found in MP4 files.
To learn more about Annex B, check for instance )
Key frames (IRAP) require inclusion of a SPS (Sequence Parameter Set)
Enum value is the fourcc for 'hev1' (the WebCodec string assigned to this codec) in big endian.
#### `VP8` = 0x76703038
VP8
See
Enum value is the fourcc for 'vp08' (the WebCodec string assigned to this codec) in big endian.
#### `VP9` = 0x76703039
VP9
See
Enum value is the fourcc for 'vp09' (the WebCodec string assigned to this codec) in big endian.
## Arrow datatype
```
UInt32
```
## API reference links
* π [C++ API docs for `VideoCodec`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `VideoCodec`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VideoCodec)
* π¦ [Rust API docs for `VideoCodec`](https://docs.rs/rerun/latest/rerun/components/enum.VideoCodec.html)
## Used by
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
# VoxelIndex
Integer index of a voxel in a sparse 3D voxel grid.
The voxel center in local grid coordinates is `(index + 0.5) * voxel_size`.
## Rerun datatype
[`IVec3D`](https://rerun.io/docs/reference/types/datatypes/ivec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Int32)
```
## API reference links
* π [C++ API docs for `VoxelIndex`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1VoxelIndex.html)
* π [Python API docs for `VoxelIndex`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VoxelIndex)
* π¦ [Rust API docs for `VoxelIndex`](https://docs.rs/rerun/latest/rerun/components/struct.VoxelIndex.html)
## Used by
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# VideoSample
Video sample data (also known as "video chunk").
Each video sample must contain enough data for exactly one video frame
(this restriction may be relaxed in the future for some codecs).
Keyframes may require additional data, for details see [`components.VideoCodec`](https://rerun.io/docs/reference/types/components/video_codec).
## Rerun datatype
[`Blob`](https://rerun.io/docs/reference/types/datatypes/blob.md)
## Arrow datatype
```
List(non-null UInt8)
```
## API reference links
* π [C++ API docs for `VideoSample`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1VideoSample.html)
* π [Python API docs for `VideoSample`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VideoSample)
* π¦ [Rust API docs for `VideoSample`](https://docs.rs/rerun/latest/rerun/components/struct.VideoSample.html)
## Used by
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
# CellSize
The metric size of one grid cell in local scene units.
E.g. for 2D grid maps, this is the physical size represented by a single pixel or cell.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `CellSize`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1CellSize.html)
* π [Python API docs for `CellSize`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.CellSize)
* π¦ [Rust API docs for `CellSize`](https://docs.rs/rerun/latest/rerun/components/struct.CellSize.html)
## Used by
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
# GammaCorrection
A gamma correction value to be used with a scalar value or color.
Used to adjust the gamma of a color or scalar value between 0 and 1 before rendering.
`new_value = old_value ^ gamma`
Must be a positive number.
Defaults to 1.0 unless otherwise specified.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `GammaCorrection`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1GammaCorrection.html)
* π [Python API docs for `GammaCorrection`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.GammaCorrection)
* π¦ [Rust API docs for `GammaCorrection`](https://docs.rs/rerun/latest/rerun/components/struct.GammaCorrection.html)
# ShowLabels
Whether the entity's [`components.Text`](https://rerun.io/docs/reference/types/components/text) label is shown.
The main purpose of this component existing separately from the labels themselves
is to be overridden when desired, to allow hiding and showing from the viewer and
blueprints.
## Rerun datatype
[`Bool`](https://rerun.io/docs/reference/types/datatypes/bool.md)
## Arrow datatype
```
Boolean
```
## API reference links
* π [C++ API docs for `ShowLabels`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ShowLabels.html)
* π [Python API docs for `ShowLabels`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ShowLabels)
* π¦ [Rust API docs for `ShowLabels`](https://docs.rs/rerun/latest/rerun/components/struct.ShowLabels.html)
## Used by
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
* [`TransformAxes3D`](https://rerun.io/docs/reference/types/archetypes/transform_axes3d.md)
# MagnificationFilter
Filter used when a single texel/pixel of an image is displayed larger than a single screen pixel.
This happens when zooming into an image, when displaying a low-resolution image in a large area,
or when viewing an image up close in 3D space.
## Variants
#### `Nearest` = 1
Show the nearest pixel value.
This will give a blocky appearance when the image is scaled up.
Used as default when rendering 2D images.
#### `Linear` = 2
Linearly interpolate the nearest neighbors, creating a smoother look when the image is scaled up.
Used as default for mesh rendering.
#### `Bicubic` = 3
Bicubic interpolation using a Catmull-Rom spline, creating the smoothest look when the image is scaled up.
This is computationally more expensive than linear filtering but produces sharper results with less blurring.
Unlike bilinear filtering, this avoids cross-shaped artifacts at texel boundaries.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `MagnificationFilter`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1components.html)
* π [Python API docs for `MagnificationFilter`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.MagnificationFilter)
* π¦ [Rust API docs for `MagnificationFilter`](https://docs.rs/rerun/latest/rerun/components/enum.MagnificationFilter.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
# Opacity
Degree of transparency ranging from 0.0 (fully transparent) to 1.0 (fully opaque).
The final opacity value may be a result of multiplication with alpha values as specified by other color sources.
Unless otherwise specified, the default value is 1.
## Rerun datatype
[`Float32`](https://rerun.io/docs/reference/types/datatypes/float32.md)
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `Opacity`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Opacity.html)
* π [Python API docs for `Opacity`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Opacity)
* π¦ [Rust API docs for `Opacity`](https://docs.rs/rerun/latest/rerun/components/struct.Opacity.html)
## Used by
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md)
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md)
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# Blob
A binary blob of data.
## Rerun datatype
[`Blob`](https://rerun.io/docs/reference/types/datatypes/blob.md)
## Arrow datatype
```
List(non-null UInt8)
```
## API reference links
* π [C++ API docs for `Blob`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Blob.html)
* π [Python API docs for `Blob`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Blob)
* π¦ [Rust API docs for `Blob`](https://docs.rs/rerun/latest/rerun/components/struct.Blob.html)
## Used by
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md)
* [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`McapMessage`](https://rerun.io/docs/reference/types/archetypes/mcap_message.md)
* [`McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema.md)
# IsKeyframe
Whether a [`components.VideoSample`](https://rerun.io/docs/reference/types/components/video_sample) contains a keyframe (also known as a sync sample or IDR).
A keyframe in this sense must be _decoder re-entrant_: a decoder must be able to start
decoding the stream from this sample alone, with no prior decoder state.
Not every intra-coded frame qualifies. Some codecs have intra-only frames that may
still reference existing decoder state and are therefore not valid sync points.
See [`components.VideoCodec`](https://rerun.io/docs/reference/types/components/video_codec) for the codec-specific definition of a keyframe.
## Rerun datatype
[`Bool`](https://rerun.io/docs/reference/types/datatypes/bool.md)
## Arrow datatype
```
Boolean
```
## API reference links
* π [C++ API docs for `IsKeyframe`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1IsKeyframe.html)
* π [Python API docs for `IsKeyframe`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.IsKeyframe)
* π¦ [Rust API docs for `IsKeyframe`](https://docs.rs/rerun/latest/rerun/components/struct.IsKeyframe.html)
## Used by
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
# Translation3D
A translation vector in 3D space.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Translation3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Translation3D.html)
* π [Python API docs for `Translation3D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Translation3D)
* π¦ [Rust API docs for `Translation3D`](https://docs.rs/rerun/latest/rerun/components/struct.Translation3D.html)
## Used by
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# ValueRange
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
Range of expected or valid values, specifying a lower and upper bound.
## Rerun datatype
[`Range1D`](https://rerun.io/docs/reference/types/datatypes/range1d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float64)
```
## API reference links
* π [C++ API docs for `ValueRange`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ValueRange.html)
* π [Python API docs for `ValueRange`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ValueRange)
* π¦ [Rust API docs for `ValueRange`](https://docs.rs/rerun/latest/rerun/components/struct.ValueRange.html)
## Used by
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# VoxelSize
The scene-unit dimensions of one voxel in a sparse 3D voxel grid.
Each component is the size of a voxel along the corresponding local grid axis.
All components must be finite and positive.
## Rerun datatype
[`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `VoxelSize`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1VoxelSize.html)
* π [Python API docs for `VoxelSize`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.VoxelSize)
* π¦ [Rust API docs for `VoxelSize`](https://docs.rs/rerun/latest/rerun/components/struct.VoxelSize.html)
## Used by
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# Count
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A generic count value.
Used for counting various entities like messages, schemas, channels, etc.
## Rerun datatype
[`UInt64`](https://rerun.io/docs/reference/types/datatypes/uint64.md)
## Arrow datatype
```
UInt64
```
## API reference links
* π [C++ API docs for `Count`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1Count.html)
* π [Python API docs for `Count`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.Count)
* π¦ [Rust API docs for `Count`](https://docs.rs/rerun/latest/rerun/components/struct.Count.html)
## Used by
* [`McapStatistics`](https://rerun.io/docs/reference/types/archetypes/mcap_statistics.md)
# TransformFrameId
A string identifier for a transform frame.
Transform frames may be derived from entity paths to refer to Rerun's implicit
entity path driven hierarchy which is defined via [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d), [`archetypes.Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole) etc..
These implicit transform frames look like `tf#path/to/entity`.
Note that any [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d)s logged with both `parent_frame` and `child_frame` set
describes a relationship between these parent and child transform frames, **not** the transform frame
that the entity path may be using (defined by an [`archetypes.CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame)).
## Rerun datatype
[`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `TransformFrameId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TransformFrameId.html)
* π [Python API docs for `TransformFrameId`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TransformFrameId)
* π¦ [Rust API docs for `TransformFrameId`](https://docs.rs/rerun/latest/rerun/components/struct.TransformFrameId.html)
## Used by
* [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
# HalfSize2D
Half-size (radius) of a 2D box.
Measured in its local coordinate system.
The box extends both in negative and positive direction along each axis.
Negative sizes indicate that the box is flipped along the respective axis, but this has no effect on how it is displayed.
## Rerun datatype
[`Vec2D`](https://rerun.io/docs/reference/types/datatypes/vec2d.md)
## Arrow datatype
```
FixedSizeList(2 x non-null Float32)
```
## API reference links
* π [C++ API docs for `HalfSize2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1HalfSize2D.html)
* π [Python API docs for `HalfSize2D`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.HalfSize2D)
* π¦ [Rust API docs for `HalfSize2D`](https://docs.rs/rerun/latest/rerun/components/struct.HalfSize2D.html)
## Used by
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
# RotationAxisAngle
3D rotation represented by a rotation around a given axis.
If normalization of the rotation axis fails the rotation is treated as an invalid transform, unless the
angle is zero in which case it is treated as an identity.
## Rerun datatype
[`RotationAxisAngle`](https://rerun.io/docs/reference/types/datatypes/rotation_axis_angle.md)
## Arrow datatype
```
Struct(
"axis": non-null FixedSizeList(3 x non-null Float32)
"angle": non-null Float32
)
```
## API reference links
* π [C++ API docs for `RotationAxisAngle`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1RotationAxisAngle.html)
* π [Python API docs for `RotationAxisAngle`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.RotationAxisAngle)
* π¦ [Rust API docs for `RotationAxisAngle`](https://docs.rs/rerun/latest/rerun/components/struct.RotationAxisAngle.html)
## Used by
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
# TensorData
An N-dimensional array of numbers.
The number of dimensions and their respective lengths is specified by the `shape` field.
The dimensions are ordered from outermost to innermost. For example, in the common case of
a 2D RGB Image, the shape would be `[height, width, channel]`.
These dimensions are combined with an index to look up values from the `buffer` field,
which stores a contiguous array of typed values.
## Rerun datatype
[`TensorData`](https://rerun.io/docs/reference/types/datatypes/tensor_data.md)
## Arrow datatype
```
Struct(
"shape": non-null List(non-null UInt64)
"names": List(non-null Utf8)
"buffer": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("U8": non-null List(non-null UInt8))
2: ("U16": non-null List(non-null UInt16))
3: ("U32": non-null List(non-null UInt32))
4: ("U64": non-null List(non-null UInt64))
5: ("I8": non-null List(non-null Int8))
6: ("I16": non-null List(non-null Int16))
7: ("I32": non-null List(non-null Int32))
8: ("I64": non-null List(non-null Int64))
9: ("F16": non-null List(non-null Float16))
10: ("F32": non-null List(non-null Float32))
11: ("F64": non-null List(non-null Float64))
)
)
```
## API reference links
* π [C++ API docs for `TensorData`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1TensorData.html)
* π [Python API docs for `TensorData`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.TensorData)
* π¦ [Rust API docs for `TensorData`](https://docs.rs/rerun/latest/rerun/components/struct.TensorData.html)
## Used by
* [`BarChart`](https://rerun.io/docs/reference/types/archetypes/bar_chart.md)
* [`Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor.md)
# ClearIsRecursive
Configures how a clear operation should behave - recursive or not.
## Rerun datatype
[`Bool`](https://rerun.io/docs/reference/types/datatypes/bool.md)
## Arrow datatype
```
Boolean
```
## API reference links
* π [C++ API docs for `ClearIsRecursive`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1components_1_1ClearIsRecursive.html)
* π [Python API docs for `ClearIsRecursive`](https://ref.rerun.io/docs/python/stable/common/components#rerun.components.ClearIsRecursive)
* π¦ [Rust API docs for `ClearIsRecursive`](https://docs.rs/rerun/latest/rerun/components/struct.ClearIsRecursive.html)
## Used by
* [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)
# Spatial2DView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
For viewing spatial 2D data.
## Properties
### `background`
Configuration for the background of the view.
* `kind`: The type of the background.
* `color`: Color used for the solid background type.
### `visual_bounds`
The visible parts of the scene, in the coordinate space of the scene.
Everything within these bounds are guaranteed to be visible.
Somethings outside of these bounds may also be visible due to letterboxing.
### `spatial_information`
Configuration of spatial information shown in the view.
* `target_frame`: The target reference frame for all transformations.
* `show_axes`: Whether axes should be shown at the origin.
* `show_bounding_box`: Whether the bounding box should be shown.
### `time_ranges`
Configures which range on each timeline is shown by this view (unless specified differently per entity).
If not specified, the default is to show the latest state of each component.
If a timeline is specified more than once, the first entry will be used.
## API reference links
* π [Python API docs for `Spatial2DView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.Spatial2DView)
## Example
### Use a blueprint to customize a Spatial2DView.
```python
"""Use a blueprint to customize a Spatial2DView."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_spatial_2d", spawn=True)
# Create a spiral of points:
n = 150
angle = np.linspace(0, 10 * np.pi, n)
spiral_radius = np.linspace(0.0, 3.0, n) ** 2
positions = np.column_stack((
np.cos(angle) * spiral_radius,
np.sin(angle) * spiral_radius,
))
colors = np.dstack((
np.linspace(255, 255, n),
np.linspace(255, 0, n),
np.linspace(0, 255, n),
))[0].astype(int)
radii = np.linspace(0.01, 0.7, n)
rr.log("points", rr.Points2D(positions, colors=colors, radii=radii))
# Create a Spatial2D view to display the points.
blueprint = rrb.Blueprint(
rrb.Spatial2DView(
origin="/",
name="2D Scene",
# Set the background color
background=[105, 20, 105],
# Note that this range is smaller than the range of the points,
# so some points will not be visible.
visual_bounds=rrb.VisualBounds2D(x_range=[-5, 5], y_range=[-5, 5]),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context.md)
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md)
* [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md)
* [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)
* [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md)
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md)
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md)
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md)
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md) (if logged above active projection)
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md) (if logged above active projection)
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md) (if logged above active projection)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md) (if logged above active projection)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md) (if logged above active projection)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md) (if logged above active projection)
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md) (if logged above active projection)
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md) (if logged above active projection)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md) (if logged above active projection)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md) (if logged above active projection)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md) (if logged above active projection)
# StateTimelineView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A view for displaying state transitions over time, for use with [`archetypes.StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change).
## API reference links
* π [Python API docs for `StateTimelineView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.StateTimelineView)
## Example
### Use a blueprint to show a StateTimelineView.
```python
# Use a blueprint to show a StateTimelineView.
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_state_timeline", spawn=True)
rr.set_time("step", sequence=0)
rr.log("door", rr.StateChange(state="open"))
rr.set_time("step", sequence=1)
rr.log("door", rr.StateChange(state="closed"))
rr.set_time("step", sequence=2)
rr.log("door", rr.StateChange(state="open"))
# Create a state timeline view to display the state transitions.
blueprint = rrb.Blueprint(
rrb.StateTimelineView(
origin="/",
name="State Transitions",
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change.md)
* [`StateConfiguration`](https://rerun.io/docs/reference/types/archetypes/state_configuration.md)
# BarChartView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A bar chart view.
## Properties
### `plot_legend`
Configures the legend of the plot.
* `corner`: To what corner the legend is aligned.
* `visible`: Whether the legend is shown at all.
### `background`
Configures the background of the plot.
* `color`: Color used for the background.
* `show_grid`: Should the grid be drawn?
## API reference links
* π [Python API docs for `BarChartView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.BarChartView)
## Example
### Use a blueprint to create a BarChartView.
```python
"""Use a blueprint to show a bar chart."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_bar_chart", spawn=True)
rr.log("bar_chart", rr.BarChart([8, 4, 0, 9, 1, 4, 1, 6, 9, 0]))
# Create a bar chart view to display the chart.
blueprint = rrb.Blueprint(
rrb.BarChartView(
origin="bar_chart",
name="Bar Chart",
background=rrb.archetypes.PlotBackground(
color=[50, 0, 50, 255], show_grid=False
),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`BarChart`](https://rerun.io/docs/reference/types/archetypes/bar_chart.md)
* [`Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor.md) (for 1D tensors)
# Spatial3DView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
For viewing spatial 3D data.
## Properties
### `background`
Configuration for the background of the view.
* `kind`: The type of the background.
* `color`: Color used for the solid background type.
### `line_grid`
Configuration for the 3D line grid.
* `visible`: Whether the grid is visible.
* `spacing`: Space between grid lines spacing of one line to the next in scene units.
* `plane`: In what plane the grid is drawn.
* `stroke_width`: How thick the lines should be in ui units.
* `color`: Color used for the grid.
### `spatial_information`
Configuration of debug drawing in the 3D view.
* `target_frame`: The target reference frame for all transformations.
* `show_axes`: Whether axes should be shown at the origin.
* `show_bounding_box`: Whether the bounding box should be shown.
### `eye_controls`
Configuration for the 3D eye
* `kind`: The kind of the eye for the spatial 3D view.
* `position`: The cameras current position.
* `look_target`: The position the camera is currently looking at.
* `eye_up`: The up-axis of the eye itself, in world-space.
* `speed`: Translation speed of the eye in the view (when using WASDQE keys to move in the 3D scene).
* `tracking_entity`: Currently tracked entity.
* `spin_speed`: What speed, if any, the camera should spin around the eye-up axis.
### `time_ranges`
Configures which range on each timeline is shown by this view (unless specified differently per entity).
If not specified, the default is to show the latest state of each component.
If a timeline is specified more than once, the first entry will be used.
## API reference links
* π [Python API docs for `Spatial3DView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.Spatial3DView)
## Example
### Use a blueprint to customize a Spatial3DView.
```python
"""Use a blueprint to customize a Spatial3DView."""
from numpy.random import default_rng
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_spatial_3d", spawn=True)
# Create some random points.
rng = default_rng(12345)
positions = rng.uniform(-5, 5, size=[50, 3])
colors = rng.uniform(0, 255, size=[50, 3])
radii = rng.uniform(0.1, 0.5, size=[50])
rr.log("points", rr.Points3D(positions, colors=colors, radii=radii))
rr.log("box", rr.Boxes3D(half_sizes=[5, 5, 5], colors=0))
# Create a Spatial3D view to display the points.
blueprint = rrb.Blueprint(
rrb.Spatial3DView(
origin="/",
name="3D Scene",
# Set the background color to light blue.
background=[100, 149, 237],
# Configure the eye controls.
eye_controls=rrb.EyeControls3D(
position=(0.0, 0.0, 2.0),
look_target=(0.0, 2.0, 0.0),
eye_up=(-1.0, 0.0, 0.0),
spin_speed=0.2,
kind=rrb.Eye3DKind.FirstPerson,
speed=20.0,
),
# Configure the line grid.
line_grid=rrb.LineGrid3D(
# The grid is enabled by default, but you can hide it.
visible=True,
spacing=0.1, # Makes the grid more fine-grained.
# By default, the plane is inferred from view coordinates setup,
# but you can set arbitrary planes.
plane=rr.components.Plane3D.XY.with_distance(-5.0),
stroke_width=2.0, # Makes the grid lines twice as thick as usual.
color=[
255,
255,
255,
128,
], # Colors the grid a half-transparent white.
),
spatial_information=rrb.SpatialInformation(
target_frame="tf#/",
show_axes=True,
show_bounding_box=True,
),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context.md)
* [`Arrows3D`](https://rerun.io/docs/reference/types/archetypes/arrows3d.md)
* [`Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d.md)
* [`Boxes3D`](https://rerun.io/docs/reference/types/archetypes/boxes3d.md)
* [`Capsules3D`](https://rerun.io/docs/reference/types/archetypes/capsules3d.md)
* [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)
* [`CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame.md)
* [`Cylinders3D`](https://rerun.io/docs/reference/types/archetypes/cylinders3d.md)
* [`Ellipsoids3D`](https://rerun.io/docs/reference/types/archetypes/ellipsoids3d.md)
* [`GridMap`](https://rerun.io/docs/reference/types/archetypes/grid_map.md)
* [`InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d.md)
* [`LineStrips3D`](https://rerun.io/docs/reference/types/archetypes/line_strips3d.md)
* [`Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d.md)
* [`Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole.md)
* [`Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d.md)
* [`Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d.md)
* [`TransformAxes3D`](https://rerun.io/docs/reference/types/archetypes/transform_axes3d.md)
* [`ViewCoordinates`](https://rerun.io/docs/reference/types/archetypes/view_coordinates.md)
* [`VoxelGridMap`](https://rerun.io/docs/reference/types/archetypes/voxel_grid_map.md)
* [`Arrows2D`](https://rerun.io/docs/reference/types/archetypes/arrows2d.md) (if logged under a projection)
* [`AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video.md) (if logged under a projection)
* [`Boxes2D`](https://rerun.io/docs/reference/types/archetypes/boxes2d.md) (if logged under a projection)
* [`DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image.md) (if logged under a projection)
* [`Ellipses2D`](https://rerun.io/docs/reference/types/archetypes/ellipses2d.md) (if logged under a projection)
* [`EncodedDepthImage`](https://rerun.io/docs/reference/types/archetypes/encoded_depth_image.md) (if logged under a projection)
* [`EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image.md) (if logged under a projection)
* [`Image`](https://rerun.io/docs/reference/types/archetypes/image.md) (if logged under a projection)
* [`LineStrips2D`](https://rerun.io/docs/reference/types/archetypes/line_strips2d.md) (if logged under a projection)
* [`Points2D`](https://rerun.io/docs/reference/types/archetypes/points2d.md) (if logged under a projection)
* [`SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image.md) (if logged under a projection)
* [`VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference.md) (if logged under a projection)
* [`VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream.md) (if logged under a projection)
# TextDocumentView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A view of a single text document, for use with [`archetypes.TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document).
## Properties
### `format_options`
Formatting options for the text document view.
* `monospace`: Whether to use a monospace font for the document body.
* `word_wrap`: Whether to wrap long lines in the document body.
## API reference links
* π [Python API docs for `TextDocumentView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.TextDocumentView)
## Example
### Use a blueprint to show a text document.
```python
"""Use a blueprint to show a text document."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_text_document", spawn=True)
rr.log(
"markdown",
rr.TextDocument(
'''
# Hello Markdown!
[Click here to see the raw text](recording://markdown:Text).
Basic formatting:
| **Feature** | **Alternative** |
| ----------------- | --------------- |
| Plain | |
| *italics* | _italics_ |
| **bold** | __bold__ |
| ~~strikethrough~~ | |
| `inline code` | |
----------------------------------
## Support
- [x] [Commonmark](https://commonmark.org/help/) support
- [x] GitHub-style strikethrough, tables, and checkboxes
- Basic syntax highlighting for:
- [x] C and C++
- [x] Python
- [x] Rust
- [ ] Other languages
## Links
You can link to [an entity](recording://markdown),
a [specific instance of an entity](recording://markdown[#0]),
or a [specific component](recording://markdown:Text).
Of course you can also have [normal https links](https://github.com/rerun-io/rerun), e.g. .
## Image
'''.strip(),
media_type=rr.MediaType.MARKDOWN,
),
)
# Create a text view that displays the markdown.
blueprint = rrb.Blueprint(rrb.TextDocumentView(origin="markdown", name="Markdown example"), collapse_panels=True)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`TextDocument`](https://rerun.io/docs/reference/types/archetypes/text_document.md)
# TimeSeriesView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A time series view for scalars over time, for use with [`archetypes.Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars).
## Properties
### `axis_x`
Configures the horizontal axis of the plot.
* `link`: How should the horizontal/X/time axis be linked across multiple plots?
* `view_range`: The view range of the horizontal/X/time axis.
* `zoom_lock`: If enabled, the X axis range will remain locked to the specified range when zooming.
### `axis_y`
Configures the vertical axis of the plot.
* `range`: The range of the axis.
* `zoom_lock`: If enabled, the Y axis range will remain locked to the specified range when zooming.
### `plot_legend`
Configures the legend of the plot.
* `corner`: To what corner the legend is aligned.
* `visible`: Whether the legend is shown at all.
### `background`
Configures the background of the plot.
* `color`: Color used for the background.
* `show_grid`: Should the grid be drawn?
### `time_ranges`
Configures which range on each timeline is shown by this view (unless specified differently per entity).
If not specified, the default is to show the entire timeline.
If a timeline is specified more than once, the first entry will be used.
## API reference links
* π [Python API docs for `TimeSeriesView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.TimeSeriesView)
## Example
### Use a blueprint to customize a TimeSeriesView.
```python
"""Use a blueprint to customize a TimeSeriesView."""
import math
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_timeseries", spawn=True)
# Log some trigonometric functions
rr.log(
"trig/sin",
rr.SeriesLines(colors=[255, 0, 0], names="sin(0.01t)"),
static=True,
)
rr.log(
"trig/cos",
rr.SeriesLines(colors=[0, 255, 0], names="cos(0.01t)"),
static=True,
)
rr.log(
"trig/cos_scaled",
rr.SeriesLines(colors=[0, 0, 255], names="cos(0.01t) scaled"),
static=True,
)
for t in range(int(math.pi * 4 * 100.0)):
rr.set_time("timeline0", sequence=t)
rr.set_time("timeline1", duration=t)
rr.log("trig/sin", rr.Scalars(math.sin(float(t) / 100.0)))
rr.log("trig/cos", rr.Scalars(math.cos(float(t) / 100.0)))
rr.log("trig/cos_scaled", rr.Scalars(math.cos(float(t) / 100.0) * 2.0))
# Create a TimeSeries View
blueprint = rrb.Blueprint(
rrb.Vertical(
contents=[
rrb.TimeSeriesView(
origin="/trig",
# Set a custom Y axis.
axis_y=rrb.ScalarAxis(range=(-1.0, 1.0), zoom_lock=True),
# Configure the legend.
plot_legend=rrb.PlotLegend(visible=False),
# Set time different time ranges for different timelines.
time_ranges=[
# Sliding window depending on the time cursor for the
# first timeline.
rrb.VisibleTimeRange(
"timeline0",
start=rrb.TimeRangeBoundary.cursor_relative(seq=-100),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
# Time range from some point to the end of the timeline
# for the second timeline.
rrb.VisibleTimeRange(
"timeline1",
start=rrb.TimeRangeBoundary.absolute(seconds=300.0),
end=rrb.TimeRangeBoundary.infinite(),
),
],
),
rrb.TimeSeriesView(
origin="/trig",
axis_x=rrb.TimeAxis(
view_range=rr.TimeRange(
start=rrb.TimeRangeBoundary.cursor_relative(
seconds=-100
),
end=rrb.TimeRangeBoundary.cursor_relative(seconds=100),
),
zoom_lock=True,
),
# Configure the legend.
plot_legend=rrb.PlotLegend(visible=True),
background=rrb.archetypes.PlotBackground(
color=[128, 128, 128], show_grid=False
),
),
]
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`Clear`](https://rerun.io/docs/reference/types/archetypes/clear.md)
* [`Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars.md)
* [`SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines.md)
* [`SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points.md)
# TensorView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A view on a tensor of any dimensionality.
## Properties
### `slice_selection`
How to select the slice of the tensor to show.
* `width`: Which dimension to map to width.
* `height`: Which dimension to map to height.
* `indices`: Selected indices for all other dimensions.
* `slider`: Any dimension listed here will have a slider for the index.
### `scalar_mapping`
Configures how scalars are mapped to color.
* `mag_filter`: Filter used when zooming in on the tensor.
* `colormap`: How scalar values map to colors.
* `gamma`: Gamma exponent applied to normalized values before mapping to color.
### `view_fit`
Configures how the selected slice should fit into the view.
## API reference links
* π [Python API docs for `TensorView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.TensorView)
## Example
### Use a blueprint to create a TensorView.
```python
"""Use a blueprint to show a tensor view."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_tensor", spawn=True)
tensor = np.random.randint(0, 256, (32, 240, 320, 3), dtype=np.uint8)
rr.log("tensor", rr.Tensor(tensor, dim_names=("batch", "x", "y", "channel")))
blueprint = rrb.Blueprint(
rrb.TensorView(
origin="tensor",
name="Tensor",
# Explicitly pick which dimensions to show.
slice_selection=rrb.TensorSliceSelection(
# Use the first dimension as width.
width=1,
# Use the second dimension as height and invert it.
height=rr.TensorDimensionSelection(dimension=2, invert=True),
# Set which indices to show for the other dimensions.
indices=[
rr.TensorDimensionIndexSelection(dimension=2, index=4),
rr.TensorDimensionIndexSelection(dimension=3, index=5),
],
# Show a slider for dimension 2 only. If not specified, all
# dimensions in `indices` will have sliders.
slider=[2],
),
# Set a scalar mapping with a custom colormap, gamma and
# magnification filter.
scalar_mapping=rrb.TensorScalarMapping(
colormap="turbo", gamma=1.5, mag_filter="linear"
),
# Fill the view, ignoring aspect ratio.
view_fit="fill",
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor.md)
# DataframeView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A view to display any data in a tabular form.
Any data from the store can be shown, using a flexible, user-configurable query.
See [Dataframe queries](https://rerun.io/docs/concepts/query-and-transform/dataframe-queries) to learn more about the query model.
## Properties
### `query`
Query of the dataframe.
* `timeline`: The timeline for this query.
* `filter_by_range`: If provided, only rows whose timestamp is within this range will be shown.
* `filter_is_not_null`: If provided, only show rows which contains a logged event for the specified component.
* `apply_latest_at`: Should empty cells be filled with latest-at queries?
* `select`: Selected columns. If unset, only the active timeline and all component columns are selected.
* `entity_order`: The order of entity path column groups. If unset, the default order is used.
* `auto_scroll`: Whether to auto-scroll to track the time cursor.
## API reference links
* π [Python API docs for `DataframeView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.DataframeView)
## Example
### Use a blueprint to customize a DataframeView.
```python
"""Use a blueprint to customize a DataframeView."""
import math
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_dataframe", spawn=True)
# Log some data.
for t in range(int(math.pi * 4 * 100.0)):
rr.set_time("t", duration=t)
rr.log("trig/sin", rr.Scalars(math.sin(float(t) / 100.0)))
rr.log("trig/cos", rr.Scalars(math.cos(float(t) / 100.0)))
# some sparse data
if t % 5 == 0:
rr.log("trig/tan_sparse", rr.Scalars(math.tan(float(t) / 100.0)))
# Create a Dataframe View
blueprint = rrb.Blueprint(
rrb.DataframeView(
origin="/trig",
query=rrb.archetypes.DataframeQuery(
timeline="t",
filter_by_range=(rr.TimeInt(seconds=0), rr.TimeInt(seconds=20)),
filter_is_not_null="/trig/tan_sparse:Scalar",
select=[
"t",
"log_tick",
"/trig/sin:Scalar",
"/trig/cos:Scalar",
"/trig/tan_sparse:Scalar",
],
entity_order=["/trig/cos", "/trig/sin", "/trig/tan_sparse"],
auto_scroll=True,
),
),
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
Any data can be displayed by the Dataframe view.
# TextLogView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A view of a text log, for use with [`archetypes.TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log).
## Properties
### `columns`
The columns to display in the view.
* `timeline_columns`: What timeline columns to show.
* `text_log_columns`: All columns to be displayed.
### `rows`
Filter for rows to display in the view.
### `format_options`
Formatting options for the text log view.
## API reference links
* π [Python API docs for `TextLogView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.TextLogView)
## Example
### Use a blueprint to show a TextLogView.
```python
"""Use a blueprint to show a text log."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_text_log", spawn=True)
rr.set_time("time", sequence=0)
rr.log(
"log/status", rr.TextLog("Application started.", level=rr.TextLogLevel.INFO)
)
rr.set_time("time", sequence=5)
rr.log("log/other", rr.TextLog("A warning.", level=rr.TextLogLevel.WARN))
for i in range(10):
rr.set_time("time", sequence=i)
rr.log(
"log/status",
rr.TextLog(f"Processing item {i}.", level=rr.TextLogLevel.INFO),
)
# Create a text view that displays all logs.
blueprint = rrb.Blueprint(
rrb.TextLogView(
origin="/log",
name="Text Logs",
columns=rrb.TextLogColumns(
timeline_columns=["time"],
text_log_columns=["loglevel", "entitypath", "body"],
),
rows=rrb.TextLogRows(
filter_by_log_level=["INFO", "WARN", "ERROR"],
),
format_options=rrb.TextLogFormat(
monospace_body=False,
),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`TextLog`](https://rerun.io/docs/reference/types/archetypes/text_log.md)
# MapView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A 2D map view to display geospatial primitives.
## Properties
### `zoom`
Configures the zoom level of the map view.
### `background`
Configuration for the background map of the map view.
## API reference links
* π [Python API docs for `MapView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.MapView)
## Example
### Use a blueprint to create a map view.
```python
"""Use a blueprint to customize a map view."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_map_view", spawn=True)
rr.log(
"points",
rr.GeoPoints(
lat_lon=[[47.6344, 19.1397], [47.6334, 19.1399]],
radii=rr.Radius.ui_points(20.0),
),
)
# Create a map view to display the chart.
blueprint = rrb.Blueprint(
rrb.MapView(
origin="points",
name="MapView",
zoom=16.0,
background=rrb.MapProvider.OpenStreetMap,
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`GeoLineStrings`](https://rerun.io/docs/reference/types/archetypes/geo_line_strings.md)
* [`GeoPoints`](https://rerun.io/docs/reference/types/archetypes/geo_points.md)
# GraphView
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A graph view to display time-variying, directed or undirected graph visualization.
## Properties
### `background`
Configures the background of the graph.
### `visual_bounds`
Everything within these bounds is guaranteed to be visible.
Some things outside of these bounds may also be visible due to letterboxing.
### `force_link`
Allows to control the interaction between two nodes connected by an edge.
* `enabled`: Whether the link force is enabled.
* `distance`: The target distance between two nodes.
* `iterations`: Specifies how often this force should be applied per iteration.
### `force_many_body`
A force between each pair of nodes that ressembles an electrical charge.
* `enabled`: Whether the many body force is enabled.
* `strength`: The strength of the force.
### `force_position`
Similar to gravity, this force pulls nodes towards a specific position.
* `enabled`: Whether the position force is enabled.
* `strength`: The strength of the force.
* `position`: The position where the nodes should be pulled towards.
### `force_collision_radius`
Resolves collisions between the bounding spheres, according to the radius of the nodes.
* `enabled`: Whether the collision force is enabled.
* `strength`: The strength of the force.
* `iterations`: Specifies how often this force should be applied per iteration.
### `force_center`
Tries to move the center of mass of the graph to the origin.
* `enabled`: Whether the center force is enabled.
* `strength`: The strength of the force.
## API reference links
* π [Python API docs for `GraphView`](https://ref.rerun.io/docs/python/stable/common/blueprint_views#rerun.blueprint.views.GraphView)
## Example
### Use a blueprint to create a graph view.
```python
"""Use a blueprint to customize a graph view."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_graph_view", spawn=True)
rr.log(
"simple",
rr.GraphNodes(
node_ids=["a", "b", "c"],
positions=[(0.0, 100.0), (-100.0, 0.0), (100.0, 0.0)],
labels=["A", "B", "C"],
),
)
# Create a Spatial2D view to display the points.
blueprint = rrb.Blueprint(
rrb.GraphView(
origin="/",
name="Graph",
# Note that this translates the viewbox.
visual_bounds=rrb.VisualBounds2D(
x_range=[-150, 150], y_range=[-50, 150]
),
background=rrb.archetypes.GraphBackground(color=[30, 10, 10]),
),
collapse_panels=True,
)
rr.send_blueprint(blueprint)
```
## Visualized archetypes
* [`GraphEdges`](https://rerun.io/docs/reference/types/archetypes/graph_edges.md)
* [`GraphNodes`](https://rerun.io/docs/reference/types/archetypes/graph_nodes.md)
# VoxelGridMap
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A sparse 3D voxel grid map with grid indices and voxel dimensions.
This archetype is intended for 3D occupancy maps and other volumetric data
represented as a sparse grid of voxels with scene-unit dimensions along the local X/Y/Z axes.
The minimum corner of the voxel with `[0, 0, 0]` index is located at the origin of the entity's coordinate frame
and can have an additional offset from there through the optional translation and rotation fields.
A voxel center is at `(index + 0.5) * voxel_size` in local grid coordinates (i.e. relative to the minimum corner).
## Fields
### Required
* `voxel_indices`: [`VoxelIndex`](https://rerun.io/docs/reference/types/components/voxel_index.md)
* `voxel_size`: [`VoxelSize`](https://rerun.io/docs/reference/types/components/voxel_size.md)
### Optional
* `values`: [`VoxelValue`](https://rerun.io/docs/reference/types/components/voxel_value.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `translation`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `rotation_axis_angle`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternion`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `value_range`: [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md)
* `colormap`: [`Colormap`](https://rerun.io/docs/reference/types/components/colormap.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `VoxelGridMap`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1VoxelGridMap.html)
* π [Python API docs for `VoxelGridMap`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.VoxelGridMap)
* π¦ [Rust API docs for `VoxelGridMap`](https://docs.rs/rerun/latest/rerun/archetypes/struct.VoxelGridMap.html)
## Example
### Simple sparse voxel grid map
```python
"""Log a simple sparse voxel grid map."""
import numpy as np
import rerun as rr
voxel_indices = np.array(
[
[-1, 0, 0],
[1, 0, 0],
[1, 1, 0],
[3, 0, 0],
[3, 0, 1],
[4, 0, 1],
],
dtype=np.int32,
)
values = np.array([0.0, 0.2, 0.4, 0.6, 0.8, 1.0], dtype=np.float32)
rr.init("rerun_example_voxel_grid_map_simple", spawn=True)
rr.log(
"world/voxels",
rr.VoxelGridMap(
voxel_indices,
voxel_size=[0.25, 0.25, 0.25],
values=values,
value_range=[0.0, 1.0],
colormap=rr.components.Colormap.Turbo,
translation=[-0.5, -0.5, 0.0],
),
)
```
# RecordingInfo
A list of properties associated with a recording.
## Fields
### Optional
* `start_time`: [`Timestamp`](https://rerun.io/docs/reference/types/components/timestamp.md)
* `name`: [`Name`](https://rerun.io/docs/reference/types/components/name.md)
## Can be shown in
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `RecordingInfo`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1RecordingInfo.html)
* π [Python API docs for `RecordingInfo`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.RecordingInfo)
* π¦ [Rust API docs for `RecordingInfo`](https://docs.rs/rerun/latest/rerun/archetypes/struct.RecordingInfo.html)
## Example
### Setting recording properties
```python
"""Sets the recording properties."""
import rerun as rr
rec = rr.RecordingStream("rerun_example_recording_properties")
rec.spawn()
# Overwrites the name from above.
rec.send_recording_name("My recording")
# Start time is set automatically, but we can overwrite it at any time.
rec.send_recording_start_time_nanos(1742539110661000000)
# Adds a user-defined property to the recording, using an existing Rerun type.
rec.send_property(
"camera_left",
rr.archetypes.Points3D([[1.0, 0.1, 1.0]]),
)
# Adds another property, this time with user-defined data.
rec.send_property(
"situation",
rr.AnyValues(
confidences=[0.3, 0.4, 0.5, 0.6],
traffic="low",
weather="sunny",
),
)
# Properties, including the name, can be overwritten at any time.
rec.send_recording_name("My episode")
```
# Asset3D
A prepacked 3D asset (`.gltf`, `.glb`, `.obj`, `.stl`, etc.).
See also [`archetypes.Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d).
If there are multiple [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) instances logged to the same entity as a mesh,
an instance of the mesh will be drawn for each transform.
## Fields
### Required
* `blob`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
### Recommended
* `media_type`: [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
### Optional
* `albedo_factor`: [`AlbedoFactor`](https://rerun.io/docs/reference/types/components/albedo_factor.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Asset3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Asset3D.html)
* π [Python API docs for `Asset3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Asset3D)
* π¦ [Rust API docs for `Asset3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Asset3D.html)
## Example
### Simple 3D asset
```python
"""Log a simple 3D asset."""
import sys
import rerun as rr
if len(sys.argv) < 2:
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset3d", spawn=True)
rr.log(
"world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True
) # Set an up-axis
rr.log("world/asset", rr.Asset3D(path=sys.argv[1]))
```
# LineStrips2D
2D line strips with positions and optional colors, radii, labels, etc.
## Fields
### Required
* `strips`: [`LineStrip2D`](https://rerun.io/docs/reference/types/components/line_strip2d.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `LineStrips2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1LineStrips2D.html)
* π [Python API docs for `LineStrips2D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.LineStrips2D)
* π¦ [Rust API docs for `LineStrips2D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.LineStrips2D.html)
## Examples
### line_strips2d_simple
```python
"""Log a simple line strip."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_line_strip2d", spawn=True)
rr.log(
"strip",
rr.LineStrips2D([[[0, 0], [2, 1], [4, -1], [6, 0]]]),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 7], y_range=[-3, 3])
)
)
```
### line_strips2d_segments_simple
```python
"""Log a couple 2D line segments using 2D line strips."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_line_segments2d", spawn=True)
rr.log(
"segments",
rr.LineStrips2D(np.array([[[0, 0], [2, 1]], [[4, -1], [6, 0]]])),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 7], y_range=[-3, 3])
)
)
```
### line_strips2d_batch
```python
"""Log a batch of 2D line strips."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_line_strip2d_batch", spawn=True)
rr.log(
"strips",
rr.LineStrips2D(
[
[[0, 0], [2, 1], [4, -1], [6, 0]],
[[0, 3], [1, 4], [2, 2], [3, 4], [4, 2], [5, 4], [6, 3]],
],
colors=[[255, 0, 0], [0, 255, 0]],
radii=[0.025, 0.005],
labels=["one strip here", "and one strip there"],
),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 7], y_range=[-3, 6])
)
)
```
### Lines with scene & UI radius each
```python
"""Log lines with ui points & scene unit radii."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_line_strip2d_ui_radius", spawn=True)
# A blue line with a scene unit radii of 0.01.
points = [[0, 0], [0, 1], [1, 0], [1, 1]]
rr.log(
"scene_unit_line",
rr.LineStrips2D(
[points],
# By default, radii are interpreted as world-space units.
radii=0.01,
colors=[0, 0, 255],
),
)
# A red line with a ui point radii of 5.
# UI points are independent of zooming in Views, but are sensitive to the
# application UI scaling.
# For 100% ui scaling, UI points are equal to pixels.
points = [[3, 0], [3, 1], [4, 0], [4, 1]]
rr.log(
"ui_points_line",
rr.LineStrips2D(
[points],
# rr.Radius.ui_points produces radii that the viewer interprets
# as given in ui points.
radii=rr.Radius.ui_points(5.0),
colors=[255, 0, 0],
),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 5], y_range=[-1, 2])
)
)
```
# Transform3D
A transform between two 3D spaces, i.e. a pose.
From the point of view of the entity's coordinate system,
all components are applied in the inverse order they are listed here.
E.g. if both a translation and a mat3x3 transform are present,
the 3x3 matrix is applied first, followed by the translation.
Whenever you log this archetype, the state of the resulting transform relationship is fully reset to the new archetype.
This means that if you first log a transform with only a translation, and then log one with only a rotation,
it will be resolved to a transform with only a rotation.
(This is unlike how we usually apply latest-at semantics on an archetype where we take the latest state of any component independently)
For transforms that affect only a single entity and do not propagate along the entity tree refer to [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d).
## Fields
### Optional
* `translation`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `rotation_axis_angle`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternion`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `scale`: [`Scale3D`](https://rerun.io/docs/reference/types/components/scale3d.md)
* `mat3x3`: [`TransformMat3x3`](https://rerun.io/docs/reference/types/components/transform_mat3x3.md)
* `relation`: [`TransformRelation`](https://rerun.io/docs/reference/types/components/transform_relation.md)
* `child_frame`: [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
* `parent_frame`: [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Transform3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Transform3D.html)
* π [Python API docs for `Transform3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Transform3D)
* π¦ [Rust API docs for `Transform3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Transform3D.html)
## Examples
### Variety of 3D transforms
```python
"""Log different transforms between three arrows."""
from math import pi
import rerun as rr
from rerun.datatypes import Angle, RotationAxisAngle
rr.init("rerun_example_transform3d", spawn=True)
arrow = rr.Arrows3D(origins=[0, 0, 0], vectors=[0, 1, 0])
rr.log("base", arrow)
rr.log("base/translated", rr.Transform3D(translation=[1, 0, 0]))
rr.log("base/translated", arrow)
rr.log(
"base/rotated_scaled",
rr.Transform3D(
rotation=RotationAxisAngle(axis=[0, 0, 1], angle=Angle(rad=pi / 4)),
scale=2,
),
)
rr.log("base/rotated_scaled", arrow)
```
### Transform hierarchy
```python
"""Logs a transform hierarchy."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_transform3d_hierarchy", spawn=True)
if False:
# One space with the sun in the center, and another one with the planet.
# TODO(#5521): enable this once we have it in Rust too, so that the
# snippets compare equally
rr.send_blueprint(
rrb.Horizontal(
rrb.Spatial3DView(origin="sun"),
rrb.Spatial3DView(origin="sun/planet", contents="sun/**"),
),
)
rr.set_time("sim_time", duration=0)
# Planetary motion is typically in the XY plane.
rr.log("/", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True)
# Setup spheres, all are in the center of their own space:
rr.log(
"sun",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[1, 1, 1],
colors=[255, 200, 10],
fill_mode="solid",
),
)
rr.log(
"sun/planet",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.4, 0.4, 0.4],
colors=[40, 80, 200],
fill_mode="solid",
),
)
rr.log(
"sun/planet/moon",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.15, 0.15, 0.15],
colors=[180, 180, 180],
fill_mode="solid",
),
)
# Draw fixed paths where the planet & moon move.
d_planet = 6.0
d_moon = 3.0
angles = np.arange(0.0, 1.01, 0.01) * np.pi * 2
circle = np.array(
[np.sin(angles), np.cos(angles), angles * 0.0], dtype=np.float32
).transpose()
rr.log("sun/planet_path", rr.LineStrips3D(circle * d_planet))
rr.log("sun/planet/moon_path", rr.LineStrips3D(circle * d_moon))
# Movement via transforms.
for i in range(6 * 120):
time = i / 120.0
rr.set_time("sim_time", duration=time)
r_moon = time * 5.0
r_planet = time * 2.0
rr.log(
"sun/planet",
rr.Transform3D(
translation=[
np.sin(r_planet) * d_planet,
np.cos(r_planet) * d_planet,
0.0,
],
rotation=rr.RotationAxisAngle(axis=(1, 0, 0), degrees=20),
),
)
rr.log(
"sun/planet/moon",
rr.Transform3D(
translation=[np.cos(r_moon) * d_moon, np.sin(r_moon) * d_moon, 0.0],
relation=rr.TransformRelation.ChildFromParent,
),
)
```
### Transform hierarchy with explicit frames
```python
"""Logs a transform hierarchy using named transform frame relationships."""
import numpy as np
import rerun as rr
rr.init("rerun_example_transform3d_hierarchy_frames", spawn=True)
rr.set_time("sim_time", duration=0)
# Planetary motion is typically in the XY plane.
rr.log("/", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True)
# Setup spheres, all are in the center of their own space:
rr.log(
"sun",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[1, 1, 1],
colors=[255, 200, 10],
fill_mode="solid",
),
rr.CoordinateFrame("sun_frame"),
)
rr.log(
"planet",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.4, 0.4, 0.4],
colors=[40, 80, 200],
fill_mode="solid",
),
rr.CoordinateFrame("planet_frame"),
)
rr.log(
"moon",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.15, 0.15, 0.15],
colors=[180, 180, 180],
fill_mode="solid",
),
rr.CoordinateFrame("moon_frame"),
)
# The viewer automatically creates a 3D view at `/`. To connect it to our
# transform hierarchy, we set its coordinate frame to `sun_frame` as well.
# Alternatively, we could also set a blueprint that makes `/sun` the space
# origin.
rr.log("/", rr.CoordinateFrame("sun_frame"))
# Draw fixed paths where the planet & moon move.
d_planet = 6.0
d_moon = 3.0
angles = np.arange(0.0, 1.01, 0.01) * np.pi * 2
circle = np.array(
[np.sin(angles), np.cos(angles), angles * 0.0], dtype=np.float32
).transpose()
rr.log(
"planet_path",
rr.LineStrips3D(circle * d_planet),
rr.CoordinateFrame("sun_frame"),
)
rr.log(
"moon_path",
rr.LineStrips3D(circle * d_moon),
rr.CoordinateFrame("planet_frame"),
)
# Movement via transforms.
for i in range(6 * 120):
time = i / 120.0
rr.set_time("sim_time", duration=time)
r_moon = time * 5.0
r_planet = time * 2.0
rr.log(
"planet_transforms",
rr.Transform3D(
translation=[
np.sin(r_planet) * d_planet,
np.cos(r_planet) * d_planet,
0.0,
],
rotation=rr.RotationAxisAngle(axis=(1, 0, 0), degrees=20),
child_frame="planet_frame",
parent_frame="sun_frame",
),
)
rr.log(
"moon_transforms",
rr.Transform3D(
translation=[np.cos(r_moon) * d_moon, np.sin(r_moon) * d_moon, 0.0],
relation=rr.TransformRelation.ChildFromParent,
child_frame="moon_frame",
parent_frame="planet_frame",
),
)
```
### Update a transform over time
```python
"""
Update a transform over time.
See also the `transform3d_column_updates` example, which achieves the same
thing in a single operation.
"""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_row_updates", spawn=True)
rr.set_time("tick", sequence=0)
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
rr.TransformAxes3D(10.0),
)
for t in range(100):
rr.set_time("tick", sequence=t + 1)
rr.log(
"box",
rr.Transform3D(
translation=[0, 0, t / 10.0],
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=truncated_radians(t * 4)
),
),
)
```
### Update a transform over time, in a single operation
```python
"""
Update a transform over time, in a single operation.
This is semantically equivalent to the `transform3d_row_updates` example,
albeit much faster.
"""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_column_updates", spawn=True)
rr.set_time("tick", sequence=0)
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
rr.TransformAxes3D(10.0),
)
rr.send_columns(
"box",
indexes=[rr.TimeColumn("tick", sequence=range(1, 101))],
columns=rr.Transform3D.columns(
translation=[[0, 0, t / 10.0] for t in range(100)],
rotation_axis_angle=[
rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=truncated_radians(t * 4)
)
for t in range(100)
],
),
)
```
### Update specific properties of a transform over time
```python
"""Update specific properties of a transform over time."""
import math
import rerun as rr
def truncated_radians(deg: float) -> float:
return float(int(math.radians(deg) * 1000.0)) / 1000.0
rr.init("rerun_example_transform3d_partial_updates", spawn=True)
# Set up a 3D box.
rr.log(
"box",
rr.Boxes3D(
half_sizes=[4.0, 2.0, 1.0], fill_mode=rr.components.FillMode.Solid
),
)
# Update only the rotation of the box.
for deg in range(46):
rad = truncated_radians(deg * 4)
rr.log(
"box",
rr.Transform3D.from_fields(
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=rad
),
),
)
# Update only the position of the box.
for t in range(51):
rr.log(
"box",
rr.Transform3D.from_fields(translation=[0, 0, t / 10.0]),
)
# Update only the rotation of the box.
for deg in range(46):
rad = truncated_radians((deg + 45) * 4)
rr.log(
"box",
rr.Transform3D.from_fields(
rotation_axis_angle=rr.RotationAxisAngle(
axis=[0.0, 1.0, 0.0], radians=rad
),
),
)
# Clear all of the box's attributes.
rr.log(
"box",
rr.Transform3D.from_fields(clear_unset=True),
)
```
# InstancePoses3D
One or more transforms applied on the current entity's transform frame.
Unlike [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d), it is *not* propagated in the transform hierarchy.
If [`archetypes.CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame) is specified, it acts relative to that coordinate frame,
otherwise it is relative to the entity's implicit coordinate frame.
Whenever you log this archetype, the state of the resulting overall pose is fully reset to the new archetype.
This means that if you first log a pose with only a translation, and then log one with only a rotation,
it will be resolved to a pose with only a rotation.
(This is unlike how we usually apply latest-at semantics on an archetype where we take the latest state of any component independently)
From the point of view of the entity's coordinate system,
all components are applied in the inverse order they are listed here.
E.g. if both a translation and a mat3x3 transform are present,
the 3x3 matrix is applied first, followed by the translation.
Currently, many visualizers support only a single instance transform per entity.
Check archetype documentations for details - if not otherwise specified, only the first instance transform is applied.
Some visualizers like the mesh visualizer used for [`archetypes.Mesh3D`](https://rerun.io/docs/reference/types/archetypes/mesh3d),
will draw an object for every pose, a behavior also known as "instancing".
## Fields
### Optional
* `translations`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `rotation_axis_angles`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternions`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `scales`: [`Scale3D`](https://rerun.io/docs/reference/types/components/scale3d.md)
* `mat3x3`: [`TransformMat3x3`](https://rerun.io/docs/reference/types/components/transform_mat3x3.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `InstancePoses3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1InstancePoses3D.html)
* π [Python API docs for `InstancePoses3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.InstancePoses3D)
* π¦ [Rust API docs for `InstancePoses3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.InstancePoses3D.html)
## Examples
### Regular & instance transforms in tandem
```python
"""Log a simple 3D box with a regular & instance pose transform."""
import numpy as np
import rerun as rr
rr.init("rerun_example_instance_pose3d_combined", spawn=True)
rr.set_time("frame", sequence=0)
# Log a box and points further down in the hierarchy.
rr.log("world/box", rr.Boxes3D(half_sizes=[[1.0, 1.0, 1.0]]))
lin = np.linspace(-10, 10, 10)
z, y, x = np.meshgrid(lin, lin, lin, indexing="ij")
point_grid = np.vstack([x.flatten(), y.flatten(), z.flatten()]).T
rr.log("world/box/points", rr.Points3D(point_grid))
for i in range(180):
rr.set_time("frame", sequence=i)
# Log a regular transform which affects both the box and the points.
rr.log(
"world/box",
rr.Transform3D(
rotation_axis_angle=rr.RotationAxisAngle(
[0, 0, 1], angle=rr.Angle(deg=i * 2)
)
),
)
# Log an instance pose which affects only the box.
rr.log(
"world/box",
rr.InstancePoses3D(translations=[0, 0, abs(i * 0.1 - 5.0) - 5.0]),
)
```
### 3D mesh with instancing
```python
"""
Log a simple 3D mesh with several instance pose transforms.
This instantiate the mesh several times and will not
affect its children. This is known as mesh instancing.
"""
import rerun as rr
rr.init("rerun_example_mesh3d_instancing", spawn=True)
rr.set_time("frame", sequence=0)
rr.log(
"shape",
rr.Mesh3D(
vertex_positions=[[1, 1, 1], [-1, -1, 1], [-1, 1, -1], [1, -1, -1]],
triangle_indices=[[0, 2, 1], [0, 3, 1], [0, 3, 2], [1, 3, 2]],
vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255], [255, 255, 0]],
),
)
# This box will not be affected by its parent's instance poses!
rr.log(
"shape/box",
rr.Boxes3D(half_sizes=[[5.0, 5.0, 5.0]]),
)
for i in range(100):
rr.set_time("frame", sequence=i)
rr.log(
"shape",
rr.InstancePoses3D(
translations=[[2, 0, 0], [0, 2, 0], [0, -2, 0], [-2, 0, 0]],
rotation_axis_angles=rr.RotationAxisAngle(
[0, 0, 1], rr.Angle(deg=i * 2)
),
),
)
```
# Image
A monochrome or color image.
See also [`archetypes.DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image) and [`archetypes.SegmentationImage`](https://rerun.io/docs/reference/types/archetypes/segmentation_image).
Rerun also supports compressed images (JPEG, PNG, β¦), using [`archetypes.EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image).
For images that refer to video frames see [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference).
Compressing images or using video data instead can save a lot of bandwidth and memory.
The raw image data is stored as a single buffer of bytes in a [`components.Blob`](https://rerun.io/docs/reference/types/components/blob).
The meaning of these bytes is determined by the [`components.ImageFormat`](https://rerun.io/docs/reference/types/components/image_format) which specifies the resolution
and the pixel format (e.g. RGB, RGBA, β¦).
The order of dimensions in the underlying [`components.Blob`](https://rerun.io/docs/reference/types/components/blob) follows the typical
row-major, interleaved-pixel image format.
## Fields
### Required
* `buffer`: [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* `format`: [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
### Optional
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `magnification_filter`: [`MagnificationFilter`](https://rerun.io/docs/reference/types/components/magnification_filter.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Image`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Image.html)
* π [Python API docs for `Image`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Image)
* π¦ [Rust API docs for `Image`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Image.html)
## Examples
### image_simple
```python
"""Create and log an image."""
import numpy as np
import rerun as rr
# Create an image with numpy
image = np.zeros((200, 300, 3), dtype=np.uint8)
image[:, :, 0] = 255
image[50:150, 50:150] = (0, 255, 0)
rr.init("rerun_example_image", spawn=True)
rr.log("image", rr.Image(image))
```
### Logging images with various formats
```python
"""Create and log an image with various formats."""
import numpy as np
import rerun as rr
rr.init("rerun_example_image_formats", spawn=True)
# Simple gradient image, logged in different formats.
image = np.array(
[[[x, min(255, x + y), y] for x in range(256)] for y in range(256)],
dtype=np.uint8,
)
rr.log("image_rgb", rr.Image(image))
rr.log(
"image_green_only", rr.Image(image[:, :, 1], color_model="l")
) # Luminance only
rr.log("image_bgr", rr.Image(image[:, :, ::-1], color_model="bgr")) # BGR
# New image with Separate Y/U/V planes with 4:2:2 chroma downsampling
y = bytes([128 for y in range(256) for x in range(256)])
u = bytes([
x * 2 for y in range(256) for x in range(128)
]) # Half horizontal resolution for chroma.
v = bytes([y for y in range(256) for x in range(128)])
rr.log(
"image_yuv422",
rr.Image(
bytes=y + u + v,
width=256,
height=256,
pixel_format=rr.PixelFormat.Y_U_V16_FullRange,
),
)
```
### Image from file, PIL & OpenCV
```python
"""Log an image."""
import tempfile
import cv2
import numpy as np
from PIL import Image as PILImage
from PIL import ImageDraw
import rerun as rr
# Save a transparent PNG to a temporary file.
_, file_path = tempfile.mkstemp(suffix=".png")
image = PILImage.new("RGBA", (300, 200), color=(0, 0, 0, 0))
draw = ImageDraw.Draw(image)
draw.rectangle((0, 0, 300, 200), outline=(255, 0, 0), width=6)
draw.rounded_rectangle((50, 50, 150, 150), fill=(0, 255, 0), radius=20)
image.save(file_path)
rr.init("rerun_example_image_advanced")
# Log the image from the file.
rr.log("from_file", rr.EncodedImage(path=file_path))
# Read with Pillow and NumPy, and log the image.
image_data = np.array(PILImage.open(file_path))
rr.log("from_pillow_rgba", rr.Image(image_data))
# Drop the alpha channel from the image.
image_rgb = image_data[..., :3]
rr.log("from_pillow_rgb", rr.Image(image_rgb))
# Read with OpenCV.
image_cv = cv2.imread(file_path)
# OpenCV uses BGR ordering, we need to make this known to Rerun.
rr.log("from_opencv", rr.Image(image_cv, color_model="BGR"))
```
### Update an image over time
```python
"""
Update an image over time.
See also the `image_column_updates` example, which achieves the same
thing in a single operation.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_image_row_updates", spawn=True)
for t in range(20):
rr.set_time("time", sequence=t)
image = np.zeros((200, 300, 3), dtype=np.uint8)
image[:, :, 2] = 255
image[50:150, (t * 10) : (t * 10 + 100)] = (0, 255, 255)
rr.log("image", rr.Image(image))
```
### Update an image over time, in a single operation
```python
"""
Update an image over time, in a single operation.
This is semantically equivalent to the `image_row_updates` example,
albeit much faster.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_image_column_updates", spawn=True)
# Timeline on which the images are distributed.
times = np.arange(0, 20)
# Create a batch of images with a moving rectangle.
width, height = 300, 200
images = np.zeros((len(times), height, width, 3), dtype=np.uint8)
images[:, :, :, 2] = 255
for t in times:
images[t, 50:150, (t * 10) : (t * 10 + 100), 1] = 255
# Log the ImageFormat and indicator once, as static.
format = rr.components.ImageFormat(
width=width, height=height, color_model="RGB", channel_datatype="U8"
)
rr.log("images", rr.Image.from_fields(format=format), static=True)
# Send all images at once.
rr.send_columns(
"images",
indexes=[rr.TimeColumn("step", sequence=times)],
# Reshape the images so `Image` can tell that this is several blobs.
#
# Note that the `Image` consumes arrays of bytes, so we should ensure
# that we take a uint8 view of it. This way, this also works when
# working with datatypes other than `U8`.
columns=rr.Image.columns(
buffer=images.view(np.uint8).reshape(len(times), -1)
),
)
```
# StateConfiguration
Define the style and mapping for state values in a state timeline view.
This archetype provides configuration for how state values are displayed.
It maps raw state values to display labels, colors, and visibility.
`values`, `labels`, `colors`, and `visible` are parallel arrays: the entry
at index `i` of each describes the same state value, and only the
per-index pairing is meaningful. The four arrays should have matching
length; any secondary array (`labels`, `colors`, `visible`) that is shorter
than `values` falls back to defaults for the missing entries.
It's generally recommended to log this type as static.
The underlying data needs to be logged to the same entity path using [`archetypes.StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change).
## Fields
### Optional
* `values`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `visible`: [`Visible`](https://rerun.io/docs/reference/types/components/visible.md)
## Can be shown in
* [StateTimelineView](https://rerun.io/docs/reference/types/views/state_timeline_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `StateConfiguration`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1StateConfiguration.html)
* π [Python API docs for `StateConfiguration`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.StateConfiguration)
* π¦ [Rust API docs for `StateConfiguration`](https://docs.rs/rerun/latest/rerun/archetypes/struct.StateConfiguration.html)
## Example
### State changes with a custom style
```python
# Log a `StateChange` together with a `StateConfiguration` that customizes
# its display.
import rerun as rr
rr.init("rerun_example_state_configuration", spawn=True)
# Configure how each raw state value is displayed (label, color, visibility).
rr.log(
"door",
rr.StateConfiguration(
values=["open", "closed"],
labels=["Open", "Closed"],
colors=[0x4CAF50FF, 0xEF5350FF],
),
static=True,
)
rr.set_time("step", sequence=0)
rr.log("door", rr.StateChange(state="open"))
rr.set_time("step", sequence=1)
rr.log("door", rr.StateChange(state="closed"))
rr.set_time("step", sequence=2)
rr.log("door", rr.StateChange(state="open"))
```
# McapChannel
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A channel within an MCAP file that defines how messages are structured and encoded.
Channels in MCAP files group messages by topic and define their encoding format.
Each channel has a unique identifier and specifies the message schema and encoding used
for all messages published to that topic.
See also [`archetypes.McapMessage`](https://rerun.io/docs/reference/types/archetypes/mcap_message) for individual messages within a channel,
[`archetypes.McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema) for the data structure definitions, and the
[MCAP specification](https://mcap.dev/) for complete format details.
## Fields
### Required
* `id`: [`ChannelId`](https://rerun.io/docs/reference/types/components/channel_id.md)
* `topic`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `message_encoding`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
### Optional
* `metadata`: [`KeyValuePairs`](https://rerun.io/docs/reference/types/components/key_value_pairs.md)
## Can be shown in
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `McapChannel`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1McapChannel.html)
* π [Python API docs for `McapChannel`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.McapChannel)
* π¦ [Rust API docs for `McapChannel`](https://docs.rs/rerun/latest/rerun/archetypes/struct.McapChannel.html)
## Example
### Simple MCAP channel
```python
"""Log a simple MCAP channel definition."""
import rerun as rr
rr.init("rerun_example_mcap_channel", spawn=True)
rr.log(
"mcap/channels/camera",
rr.McapChannel(
id=1,
topic="/camera/image",
message_encoding="cdr",
metadata={"frame_id": "camera_link", "encoding": "bgr8"},
),
)
```
# Points2D
A 2D point cloud with positions and optional colors, radii, labels, etc.
## Fields
### Required
* `positions`: [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
* `keypoint_ids`: [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Points2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Points2D.html)
* π [Python API docs for `Points2D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Points2D)
* π¦ [Rust API docs for `Points2D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Points2D.html)
## Examples
### Simple 2D points
```python
"""Log some very simple points."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_points2d", spawn=True)
rr.log("points", rr.Points2D([[0, 0], [1, 1]]))
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 2], y_range=[-1, 2])
)
)
```
### Randomly distributed 2D points with varying color and radius
```python
"""Log some random points with color and radii."""
from numpy.random import default_rng
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_points2d_random", spawn=True)
rng = default_rng(12345)
positions = rng.uniform(-3, 3, size=[10, 2])
colors = rng.uniform(0, 255, size=[10, 4])
radii = rng.uniform(0, 1, size=[10])
rr.log("random", rr.Points2D(positions, colors=colors, radii=radii))
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-4, 4], y_range=[-4, 4])
)
)
```
### Log points with radii given in UI points
```python
"""Log some points with ui points & scene unit radii."""
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_points2d_ui_radius", spawn=True)
# Two blue points with scene unit radii of 0.1 and 0.3.
rr.log(
"scene_units",
rr.Points2D(
[[0, 0], [0, 1]],
# By default, radii are interpreted as world-space units.
radii=[0.1, 0.3],
colors=[0, 0, 255],
),
)
# Two red points with ui point radii of 40 and 60.
# UI points are independent of zooming in Views, but are sensitive to the
# application UI scaling.
# For 100% ui scaling, UI points are equal to pixels.
rr.log(
"ui_points",
rr.Points2D(
[[1, 0], [1, 1]],
# rr.Radius.ui_points produces radii that the viewer interprets
# as given in ui points.
radii=rr.Radius.ui_points([40.0, 60.0]),
colors=[255, 0, 0],
),
)
# Set view bounds:
rr.send_blueprint(
rrb.Spatial2DView(
visual_bounds=rrb.VisualBounds2D(x_range=[-1, 2], y_range=[-1, 2])
)
)
```
# McapStatistics
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
Recording-level statistics about an MCAP file.
This archetype contains summary information about an entire MCAP recording, including
counts of messages, schemas, channels, and other records, as well as timing information
spanning the full recording duration. It is typically logged once per recording to provide
an overview of the dataset's structure and content.
See also [`archetypes.McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel) for individual channel definitions,
[`archetypes.McapMessage`](https://rerun.io/docs/reference/types/archetypes/mcap_message) for message content, [`archetypes.McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema) for schema definitions,
and the [MCAP specification](https://mcap.dev/) for complete format details.
## Fields
### Required
* `message_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `schema_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `channel_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `attachment_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `metadata_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `chunk_count`: [`Count`](https://rerun.io/docs/reference/types/components/count.md)
* `message_start_time`: [`Timestamp`](https://rerun.io/docs/reference/types/components/timestamp.md)
* `message_end_time`: [`Timestamp`](https://rerun.io/docs/reference/types/components/timestamp.md)
### Optional
* `channel_message_counts`: [`ChannelMessageCounts`](https://rerun.io/docs/reference/types/components/channel_message_counts.md)
## Can be shown in
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `McapStatistics`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1McapStatistics.html)
* π [Python API docs for `McapStatistics`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.McapStatistics)
* π¦ [Rust API docs for `McapStatistics`](https://docs.rs/rerun/latest/rerun/archetypes/struct.McapStatistics.html)
## Example
### Simple MCAP statistics
```python
"""Log simple MCAP recording statistics."""
import rerun as rr
rr.init("rerun_example_mcap_statistics", spawn=True)
rr.log(
"mcap/statistics/recording_overview",
rr.McapStatistics(
message_count=12500,
schema_count=3,
channel_count=5,
attachment_count=2,
metadata_count=8,
chunk_count=25,
# 2024-04-01 00:00:00 UTC in nanoseconds
message_start_time=1743465600000000000,
# 2024-04-01 00:10:00 UTC in nanoseconds (10 minute recording)
message_end_time=1743466200000000000,
),
)
```
# Boxes2D
2D boxes with half-extents and optional center, colors etc.
## Fields
### Required
* `half_sizes`: [`HalfSize2D`](https://rerun.io/docs/reference/types/components/half_size2d.md)
### Recommended
* `centers`: [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Boxes2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Boxes2D.html)
* π [Python API docs for `Boxes2D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Boxes2D)
* π¦ [Rust API docs for `Boxes2D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Boxes2D.html)
## Example
### Simple 2D boxes
```python
"""Log a simple 2D Box."""
import rerun as rr
rr.init("rerun_example_box2d", spawn=True)
rr.log("simple", rr.Boxes2D(mins=[-1, -1], sizes=[2, 2]))
```
# StateChange
A state change, representing a transition of an entity into a new state.
Useful for representing discrete state machines, mode transitions, or
state changes over time. Each logged [`archetypes.StateChange`](https://rerun.io/docs/reference/types/archetypes/state_change) marks a new state
at the given time. A `null` state resets the state, showing a gap in the state timeline view.
The state timeline view displays these as horizontal colored lanes over time.
## Fields
### Required
* `state`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
## Can be shown in
* [StateTimelineView](https://rerun.io/docs/reference/types/views/state_timeline_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `StateChange`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1StateChange.html)
* π [Python API docs for `StateChange`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.StateChange)
* π¦ [Rust API docs for `StateChange`](https://docs.rs/rerun/latest/rerun/archetypes/struct.StateChange.html)
## Example
### State changes over time
```python
# Log a `StateChange`.
import rerun as rr
rr.init("rerun_example_state_change", spawn=True)
rr.set_time("step", sequence=0)
rr.log("door", rr.StateChange(state="open"))
rr.set_time("step", sequence=1)
rr.log("door", rr.StateChange(state="closed"))
rr.set_time("step", sequence=2)
rr.log("door", rr.StateChange(state="open"))
```
# LineStrips3D
3D line strips with positions and optional colors, radii, labels, etc.
## Fields
### Required
* `strips`: [`LineStrip3D`](https://rerun.io/docs/reference/types/components/line_strip3d.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `LineStrips3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1LineStrips3D.html)
* π [Python API docs for `LineStrips3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.LineStrips3D)
* π¦ [Rust API docs for `LineStrips3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.LineStrips3D.html)
## Examples
### Simple example
```python
"""Log a simple line strip."""
import rerun as rr
rr.init("rerun_example_line_strip3d", spawn=True)
points = [
[0, 0, 0],
[0, 0, 1],
[1, 0, 0],
[1, 0, 1],
[1, 1, 0],
[1, 1, 1],
[0, 1, 0],
[0, 1, 1],
]
rr.log("strip", rr.LineStrips3D([points]))
```
### Many individual segments
```python
#!/usr/bin/env python3
"""Log a simple set of line segments."""
import numpy as np
import rerun as rr
rr.init("rerun_example_line_segments3d", spawn=True)
rr.log(
"segments",
rr.LineStrips3D(
np.array(
[
[[0, 0, 0], [0, 0, 1]],
[[1, 0, 0], [1, 0, 1]],
[[1, 1, 0], [1, 1, 1]],
[[0, 1, 0], [0, 1, 1]],
],
),
),
)
```
### Many strips
```python
"""Log a batch of 3D line strips."""
import rerun as rr
rr.init("rerun_example_line_strip3d_batch", spawn=True)
rr.log(
"strips",
rr.LineStrips3D(
[
[
[0, 0, 2],
[1, 0, 2],
[1, 1, 2],
[0, 1, 2],
],
[
[0, 0, 0],
[0, 0, 1],
[1, 0, 0],
[1, 0, 1],
[1, 1, 0],
[1, 1, 1],
[0, 1, 0],
[0, 1, 1],
],
],
colors=[[255, 0, 0], [0, 255, 0]],
radii=[0.025, 0.005],
labels=["one strip here", "and one strip there"],
),
)
```
### Lines with scene & UI radius each
```python
"""Log lines with ui points & scene unit radii."""
import rerun as rr
rr.init("rerun_example_line_strip3d_ui_radius", spawn=True)
# A blue line with a scene unit radii of 0.01.
points = [[0, 0, 0], [0, 0, 1], [1, 0, 0], [1, 0, 1]]
rr.log(
"scene_unit_line",
rr.LineStrips3D(
[points],
# By default, radii are interpreted as world-space units.
radii=0.01,
colors=[0, 0, 255],
),
)
# A red line with a ui point radii of 5.
# UI points are independent of zooming in Views, but are sensitive to the
# application UI scaling.
# For 100% ui scaling, UI points are equal to pixels.
points = [[3, 0, 0], [3, 0, 1], [4, 0, 0], [4, 0, 1]]
rr.log(
"ui_points_line",
rr.LineStrips3D(
[points],
# rr.Radius.ui_points produces radii that the viewer interprets
# as given in ui points.
radii=rr.Radius.ui_points(5.0),
colors=[255, 0, 0],
),
)
```
### Time-windowed trails (e.g. Trajectories)
```python
"""Log line strips over time and view a sliding window (e.g. trajectories)."""
import math
import rerun as rr
import rerun.blueprint as rrb
def point(t: float, phase: float) -> list[float]:
# Sample a point on a helix.
angle = 0.5 * t + phase
return [math.cos(angle), math.sin(angle), 0.1 * t]
rr.init("rerun_example_line_strips3d_time_window", spawn=True)
# Configure the visible time range in the blueprint.
# You can also override this per entity.
rr.send_blueprint(
rrb.Spatial3DView(
origin="/",
time_ranges=rrb.VisibleTimeRange(
"time",
start=rrb.TimeRangeBoundary.cursor_relative(seconds=-5.0),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
)
)
# Log the line strip increments with timestamps.
for i in range(600):
t0 = i / 30.0
t1 = (i + 1) / 30.0
rr.set_time("time", duration=t1)
rr.log(
"trails",
rr.LineStrips3D(
[
[point(t0, 0.0), point(t1, 0.0)],
[point(t0, math.pi), point(t1, math.pi)],
],
colors=[[255, 120, 0], [0, 180, 255]],
radii=0.02,
),
)
```
# McapMessage
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
The binary payload of a single MCAP message, without metadata.
This archetype represents only the raw message data from an MCAP file. It does not include
MCAP message metadata such as timestamps, channel IDs, sequence numbers, or publication times.
The binary payload represents sensor data, commands, or other information encoded according
to the format specified by the associated channel.
See [`archetypes.McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel) for channel definitions that specify message encoding,
[`archetypes.McapSchema`](https://rerun.io/docs/reference/types/archetypes/mcap_schema) for data structure definitions, and the
[MCAP specification](https://mcap.dev/) for complete format details.
## Fields
### Required
* `data`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
## Can be shown in
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `McapMessage`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1McapMessage.html)
* π [Python API docs for `McapMessage`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.McapMessage)
* π¦ [Rust API docs for `McapMessage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.McapMessage.html)
## Example
### Simple MCAP message
```python
"""Log a simple MCAP message with binary data."""
import rerun as rr
rr.init("rerun_example_mcap_message", spawn=True)
# Example binary message data (could be from a ROS message, protobuf, etc.)
# This represents a simple sensor reading encoded as bytes
sensor_data = (
b"sensor_reading: temperature=23.5, humidity=65.2, timestamp=1743465600"
)
rr.log(
"mcap/messages/sensor_reading",
rr.McapMessage(
data=sensor_data,
),
)
```
# SeriesPoints
Define the style properties for one or more point series (scatter plot) in a chart.
This archetype only provides styling information.
Changes over time are supported for most but not all its fields (see respective fields for details),
it's generally recommended to log this type as static.
The underlying data needs to be logged to the same entity-path using [`archetypes.Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars).
Dimensionality of the scalar arrays logged at each time point is assumed to be the same over time.
## Fields
### Required
* `markers`: [`MarkerShape`](https://rerun.io/docs/reference/types/components/marker_shape.md)
### Optional
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `names`: [`Name`](https://rerun.io/docs/reference/types/components/name.md)
* `visible_series`: [`Visible`](https://rerun.io/docs/reference/types/components/visible.md)
* `marker_sizes`: [`MarkerSize`](https://rerun.io/docs/reference/types/components/marker_size.md)
## Can be shown in
* [TimeSeriesView](https://rerun.io/docs/reference/types/views/time_series_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `SeriesPoints`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1SeriesPoints.html)
* π [Python API docs for `SeriesPoints`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.SeriesPoints)
* π¦ [Rust API docs for `SeriesPoints`](https://docs.rs/rerun/latest/rerun/archetypes/struct.SeriesPoints.html)
## Example
### Point series
```python
"""Log a scalar over time."""
from math import cos, sin, tau
import rerun as rr
rr.init("rerun_example_series_point_style", spawn=True)
# Set up plot styling:
# They are logged as static as they don't change over time and apply to all
# timelines. Log two point series under a shared root so that they show in the
# same plot by default.
rr.log(
"trig/sin",
rr.SeriesPoints(
colors=[255, 0, 0],
names="sin(0.01t)",
markers="circle",
marker_sizes=4,
),
static=True,
)
rr.log(
"trig/cos",
rr.SeriesPoints(
colors=[0, 255, 0],
names="cos(0.01t)",
markers="cross",
marker_sizes=2,
),
static=True,
)
# Log the data on a timeline called "step".
for t in range(int(tau * 2 * 10.0)):
rr.set_time("step", sequence=t)
rr.log("trig/sin", rr.Scalars(sin(float(t) / 10.0)))
rr.log("trig/cos", rr.Scalars(cos(float(t) / 10.0)))
```
# VideoFrameReference
References a single video frame.
Used to display individual video frames from a [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video).
To show an entire video, a video frame reference for each frame of the video should be logged.
See for details of what is and isn't supported.
TODO(#10422): [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference) does not yet work with [`archetypes.VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream).
## Fields
### Required
* `timestamp`: [`VideoTimestamp`](https://rerun.io/docs/reference/types/components/video_timestamp.md)
### Optional
* `video_reference`: [`EntityPath`](https://rerun.io/docs/reference/types/components/entity_path.md)
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `VideoFrameReference`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1VideoFrameReference.html)
* π [Python API docs for `VideoFrameReference`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.VideoFrameReference)
* π¦ [Rust API docs for `VideoFrameReference`](https://docs.rs/rerun/latest/rerun/archetypes/struct.VideoFrameReference.html)
## Examples
### Video with automatically determined frames
```python
"""Log a video asset using automatically determined frame references."""
import sys
import rerun as rr
if len(sys.argv) < 2:
# TODO(#7354): Only mp4 is supported for now.
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset_video_auto_frames", spawn=True)
# Log video asset which is referred to by frame references.
video_asset = rr.AssetVideo(path=sys.argv[1])
rr.log("video", video_asset, static=True)
# Send automatically determined video frame timestamps.
frame_timestamps_ns = video_asset.read_frame_timestamps_nanos()
rr.send_columns(
"video",
# Note timeline values don't have to be the same as the video timestamps.
indexes=[rr.TimeColumn("video_time", duration=1e-9 * frame_timestamps_ns)],
columns=rr.VideoFrameReference.columns_nanos(frame_timestamps_ns),
)
```
### Demonstrates manual use of video frame references
```python
"""Manual use of individual video frame references."""
import sys
import rerun as rr
import rerun.blueprint as rrb
if len(sys.argv) < 2:
# TODO(#7354): Only mp4 is supported for now.
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset_video_manual_frames", spawn=True)
# Log video asset which is referred to by frame references.
rr.log("video_asset", rr.AssetVideo(path=sys.argv[1]), static=True)
# Create two entities, showing the same video frozen at different times.
rr.log(
"frame_1s",
rr.VideoFrameReference(seconds=1.0, video_reference="video_asset"),
)
rr.log(
"frame_2s",
rr.VideoFrameReference(seconds=2.0, video_reference="video_asset"),
)
# Send blueprint that shows two 2D views next to each other.
rr.send_blueprint(
rrb.Horizontal(
rrb.Spatial2DView(origin="frame_1s"),
rrb.Spatial2DView(origin="frame_2s"),
)
)
```
# SeriesLines
Define the style properties for one or more line series in a chart.
This archetype only provides styling information.
Changes over time are supported for most but not all its fields (see respective fields for details),
it's generally recommended to log this type as static.
The underlying data needs to be logged to the same entity-path using [`archetypes.Scalars`](https://rerun.io/docs/reference/types/archetypes/scalars).
Dimensionality of the scalar arrays logged at each time point is assumed to be the same over time.
## Fields
### Optional
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `widths`: [`StrokeWidth`](https://rerun.io/docs/reference/types/components/stroke_width.md)
* `names`: [`Name`](https://rerun.io/docs/reference/types/components/name.md)
* `visible_series`: [`Visible`](https://rerun.io/docs/reference/types/components/visible.md)
* `aggregation_policy`: [`AggregationPolicy`](https://rerun.io/docs/reference/types/components/aggregation_policy.md)
* `interpolation_mode`: [`InterpolationMode`](https://rerun.io/docs/reference/types/components/interpolation_mode.md)
## Can be shown in
* [TimeSeriesView](https://rerun.io/docs/reference/types/views/time_series_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `SeriesLines`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1SeriesLines.html)
* π [Python API docs for `SeriesLines`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.SeriesLines)
* π¦ [Rust API docs for `SeriesLines`](https://docs.rs/rerun/latest/rerun/archetypes/struct.SeriesLines.html)
## Example
### Line series
```python
"""Log a scalar over time."""
from math import cos, sin, tau
import rerun as rr
rr.init("rerun_example_series_line_style", spawn=True)
# Set up plot styling:
# They are logged as static as they don't change over time and apply to
# all timelines.
# Log two lines series under a shared root so that they show in the same
# plot by default.
rr.log(
"trig/sin",
rr.SeriesLines(colors=[255, 0, 0], names="sin(0.01t)", widths=2),
static=True,
)
rr.log(
"trig/cos",
rr.SeriesLines(colors=[0, 255, 0], names="cos(0.01t)", widths=4),
static=True,
)
# Log the data on a timeline called "step".
for t in range(int(tau * 2 * 100.0)):
rr.set_time("step", sequence=t)
rr.log("trig/sin", rr.Scalars(sin(float(t) / 100.0)))
rr.log("trig/cos", rr.Scalars(cos(float(t) / 100.0)))
```
# TextLog
A log entry in a text log, comprised of a text body and its log level.
## Fields
### Required
* `text`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
### Recommended
* `level`: [`TextLogLevel`](https://rerun.io/docs/reference/types/components/text_log_level.md)
### Optional
* `color`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
## Can be shown in
* [TextLogView](https://rerun.io/docs/reference/types/views/text_log_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `TextLog`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1TextLog.html)
* π [Python API docs for `TextLog`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.TextLog)
* π¦ [Rust API docs for `TextLog`](https://docs.rs/rerun/latest/rerun/archetypes/struct.TextLog.html)
## Example
### text_log_integration
```python
"""Shows integration of Rerun's `TextLog` with the native logging interface."""
import logging
import rerun as rr
rr.init("rerun_example_text_log_integration", spawn=True)
# Log a text entry directly
rr.log(
"logs",
rr.TextLog("this entry has loglevel TRACE", level=rr.TextLogLevel.TRACE),
)
# Or log via a logging handler
logging.getLogger().addHandler(rr.LoggingHandler("logs/handler"))
logging.getLogger().setLevel(-1)
logging.info("This INFO log got added through the standard logging interface")
```
# VideoStream
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
Video stream consisting of raw video chunks.
For logging video containers like mp4, refer to [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video) and [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference).
To learn more about video support in Rerun, check the [video reference](https://rerun.io/docs/reference/video).
All components except `sample` are typically logged statically once per entity.
`sample` is then logged repeatedly for each frame on the timeline.
TODO(#10422): [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference) does not yet work with [`archetypes.VideoStream`](https://rerun.io/docs/reference/types/archetypes/video_stream).
## Fields
### Required
* `codec`: [`VideoCodec`](https://rerun.io/docs/reference/types/components/video_codec.md)
### Recommended
* `sample`: [`VideoSample`](https://rerun.io/docs/reference/types/components/video_sample.md)
### Optional
* `is_keyframe`: [`IsKeyframe`](https://rerun.io/docs/reference/types/components/is_keyframe.md)
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `VideoStream`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1VideoStream.html)
* π [Python API docs for `VideoStream`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.VideoStream)
* π¦ [Rust API docs for `VideoStream`](https://docs.rs/rerun/latest/rerun/archetypes/struct.VideoStream.html)
## Example
### Live streaming of on-the-fly encoded video
```python
"""Video encode images using av and stream them to Rerun."""
import av
import numpy as np
import numpy.typing as npt
import rerun as rr
fps = 30
duration_seconds = 4
width = 480
height = 320
ball_radius = 30
codec = rr.VideoCodec.H265 # rr.VideoCodec.H264
formats = {rr.VideoCodec.H265: "hevc", rr.VideoCodec.H264: "h264"}
encoders = {rr.VideoCodec.H265: "libx265", rr.VideoCodec.H264: "libx264"}
def create_example_video_frame(frame_i: int) -> npt.NDArray[np.uint8]:
img = np.zeros((height, width, 3), dtype=np.uint8)
for h in range(height):
img[h, :] = [
0,
int(100 * h / height),
int(200 * h / height),
] # Blue to purple gradient.
x_pos = width // 2 # Center horizontally.
y_pos = height // 2 + 80 * np.sin(2 * np.pi * frame_i / fps)
y, x = np.ogrid[:height, :width]
r_sq = (x - x_pos) ** 2 + (y - y_pos) ** 2
img[r_sq < ball_radius**2] = [255, 200, 0] # Gold color
return img
rr.init("rerun_example_video_stream_synthetic")
# Setup encoding pipeline.
av.logging.set_level(av.logging.VERBOSE)
container = av.open(
"/dev/null", "w", format=formats[codec]
) # Use AnnexB H.265 stream.
stream = container.add_stream(encoders[codec], rate=fps)
# Type narrowing
assert isinstance(stream, av.video.stream.VideoStream)
stream.width = width
stream.height = height
# TODO(#10090): Rerun Video Streams don't support b-frames yet.
# Note that b-frames are generally not recommended for low-latency streaming
# and may make logging more complex.
stream.max_b_frames = 0
# Log codec only once as static data (it naturally never changes).
# This isn't strictly necessary, but good practice.
rr.log("video_stream", rr.VideoStream(codec=codec), static=True)
# Generate frames and stream them directly to Rerun.
for frame_i in range(fps * duration_seconds):
img = create_example_video_frame(frame_i)
frame = av.VideoFrame.from_ndarray(img, format="rgb24")
for packet in stream.encode(frame):
if packet.pts is None:
continue
rr.set_time("time", duration=float(packet.pts * packet.time_base))
rr.log("video_stream", rr.VideoStream.from_fields(sample=bytes(packet)))
# Flush stream.
for packet in stream.encode():
if packet.pts is None:
continue
rr.set_time("time", duration=float(packet.pts * packet.time_base))
rr.log("video_stream", rr.VideoStream.from_fields(sample=bytes(packet)))
```
# Cylinders3D
3D cylinders with flat caps.
This archetype is for cylinder primitives defined by their axial length and radius.
For points whose radii are for visualization purposes, use [`archetypes.Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d) instead.
Orienting and placing cylinders forms a separate transform that is applied prior to [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) and [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
## Fields
### Required
* `lengths`: [`Length`](https://rerun.io/docs/reference/types/components/length.md)
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
### Recommended
* `centers`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `rotation_axis_angles`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternions`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `line_radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `fill_mode`: [`FillMode`](https://rerun.io/docs/reference/types/components/fill_mode.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Cylinders3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Cylinders3D.html)
* π [Python API docs for `Cylinders3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Cylinders3D)
* π¦ [Rust API docs for `Cylinders3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Cylinders3D.html)
## Example
### Batch of cylinders
```python
"""Log a batch of cylinders."""
import rerun as rr
rr.init("rerun_example_cylinders3d_batch", spawn=True)
rr.log(
"cylinders",
rr.Cylinders3D(
lengths=[0.0, 2.0, 4.0, 6.0, 8.0],
radii=[1.0, 0.5, 0.5, 0.5, 1.0],
colors=[
(255, 0, 0),
(188, 188, 0),
(0, 255, 0),
(0, 188, 188),
(0, 0, 255),
],
centers=[
(0.0, 0.0, 0.0),
(2.0, 0.0, 0.0),
(4.0, 0.0, 0.0),
(6.0, 0.0, 0.0),
(8.0, 0.0, 0.0),
],
rotation_axis_angles=[
rr.RotationAxisAngle(
[1.0, 0.0, 0.0],
rr.Angle(deg=float(i) * -22.5),
)
for i in range(5)
],
),
)
```
# TransformAxes3D
A visual representation of a [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
## Fields
### Required
* `axis_length`: [`AxisLength`](https://rerun.io/docs/reference/types/components/axis_length.md)
### Optional
* `show_frame`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `TransformAxes3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1TransformAxes3D.html)
* π [Python API docs for `TransformAxes3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.TransformAxes3D)
* π¦ [Rust API docs for `TransformAxes3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.TransformAxes3D.html)
## Example
### Visual representation of a transform as three arrows
```python
"""Log different transforms with visualized coordinates axes."""
import rerun as rr
rr.init("rerun_example_transform3d_axes", spawn=True)
rr.set_time("step", sequence=0)
# Set the axis lengths for all the transforms
rr.log("base", rr.Transform3D(), rr.TransformAxes3D(1.0))
# Now sweep out a rotation relative to the base
for deg in range(360):
rr.set_time("step", sequence=deg)
rr.log(
"base/rotated",
rr.Transform3D.from_fields(
rotation_axis_angle=rr.RotationAxisAngle(
axis=[1.0, 1.0, 1.0],
degrees=deg,
),
),
rr.TransformAxes3D(0.5),
)
rr.log(
"base/rotated/translated",
rr.Transform3D.from_fields(
translation=[2.0, 0, 0],
),
rr.TransformAxes3D(0.5),
)
```
# BarChart
A bar chart.
The bar heights will be the provided values, and the x coordinates of the bars will be the provided abscissa or default to the index of the provided values.
## Fields
### Required
* `values`: [`TensorData`](https://rerun.io/docs/reference/types/components/tensor_data.md)
### Optional
* `color`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `abscissa`: [`TensorData`](https://rerun.io/docs/reference/types/components/tensor_data.md)
* `widths`: [`Length`](https://rerun.io/docs/reference/types/components/length.md)
## Can be shown in
* [BarChartView](https://rerun.io/docs/reference/types/views/bar_chart_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `BarChart`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1BarChart.html)
* π [Python API docs for `BarChart`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.BarChart)
* π¦ [Rust API docs for `BarChart`](https://docs.rs/rerun/latest/rerun/archetypes/struct.BarChart.html)
## Example
### Simple bar chart
```python
"""Create and log a bar chart."""
import rerun as rr
rr.init("rerun_example_bar_chart", spawn=True)
rr.log("bar_chart", rr.BarChart([8, 4, 0, 9, 1, 4, 1, 6, 9, 0]))
rr.log(
"bar_chart_custom_abscissa",
rr.BarChart([8, 4, 0, 9, 1, 4], abscissa=[0, 1, 3, 4, 7, 11]),
)
rr.log(
"bar_chart_custom_abscissa_and_widths",
rr.BarChart(
[8, 4, 0, 9, 1, 4],
abscissa=[0, 1, 3, 4, 7, 11],
widths=[1, 2, 1, 3, 4, 1],
),
)
```
# DepthImage
A depth image, i.e. as captured by a depth camera.
Each pixel corresponds to a depth value in units specified by [`components.DepthMeter`](https://rerun.io/docs/reference/types/components/depth_meter).
## Fields
### Required
* `buffer`: [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* `format`: [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
### Optional
* `meter`: [`DepthMeter`](https://rerun.io/docs/reference/types/components/depth_meter.md)
* `colormap`: [`Colormap`](https://rerun.io/docs/reference/types/components/colormap.md)
* `depth_range`: [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md)
* `point_fill_ratio`: [`FillRatio`](https://rerun.io/docs/reference/types/components/fill_ratio.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `magnification_filter`: [`MagnificationFilter`](https://rerun.io/docs/reference/types/components/magnification_filter.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `DepthImage`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1DepthImage.html)
* π [Python API docs for `DepthImage`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.DepthImage)
* π¦ [Rust API docs for `DepthImage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.DepthImage.html)
## Examples
### Simple example
```python
"""Create and log a depth image."""
import numpy as np
import rerun as rr
depth_image = 65535 * np.ones((200, 300), dtype=np.uint16)
depth_image[50:150, 50:150] = 20000
depth_image[130:180, 100:280] = 45000
rr.init("rerun_example_depth_image_simple", spawn=True)
# Log the tensor, assigning names to each dimension
rr.log("depth", rr.DepthImage(depth_image, meter=10_000.0))
```
### Depth to 3D example
```python
"""Create and log a depth image and pinhole camera."""
import numpy as np
import rerun as rr
depth_image = 65535 * np.ones((200, 300), dtype=np.uint16)
depth_image[50:150, 50:150] = 20000
depth_image[130:180, 100:280] = 45000
rr.init("rerun_example_depth_image_3d", spawn=True)
# If we log a pinhole camera model, the depth gets automatically
# back-projected to 3D
rr.log(
"world/camera",
rr.Pinhole(
width=depth_image.shape[1],
height=depth_image.shape[0],
focal_length=200,
),
)
# Log the tensor.
rr.log(
"world/camera/depth",
rr.DepthImage(depth_image, meter=10_000.0, colormap="viridis"),
)
```
# CoordinateFrame
Specifies the coordinate frame for an entity.
If not specified, the coordinate frame uses an implicit frame derived from the entity path.
The implicit frame's name is `tf#/your/entity/path` and has an identity transform connection to its parent path.
To learn more about transforms see [Spaces & Transforms](https://rerun.io/docs/concepts/spaces-and-transforms) in the reference.
## Fields
### Required
* `frame`: [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `CoordinateFrame`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1CoordinateFrame.html)
* π [Python API docs for `CoordinateFrame`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.CoordinateFrame)
* π¦ [Rust API docs for `CoordinateFrame`](https://docs.rs/rerun/latest/rerun/archetypes/struct.CoordinateFrame.html)
## Examples
### Change coordinate frame to different built-in frames
```python
"""Demonstrates using explicit `CoordinateFrame` with implicit transforms."""
import rerun as rr
rr.init("rerun_example_transform3d_hierarchy", spawn=True)
rr.set_time("time", sequence=0)
rr.log(
"red_box",
rr.Boxes3D(half_sizes=[0.5, 0.5, 0.5], colors=[255, 0, 0]),
# Use Transform3D to place the box, so we actually change the underlying
# coordinate frame and not just the box's pose.
rr.Transform3D(translation=[2.0, 0.0, 0.0]),
)
rr.log(
"blue_box",
rr.Boxes3D(half_sizes=[0.5, 0.5, 0.5], colors=[0, 0, 255]),
# Use Transform3D to place the box, so we actually change the underlying
# coordinate frame and not just the box's pose.
rr.Transform3D(translation=[-2.0, 0.0, 0.0]),
)
rr.log("point", rr.Points3D([0.0, 0.0, 0.0], radii=0.5))
# Change where the point is located by cycling through its coordinate frame.
for t, frame_id in enumerate(["tf#/red_box", "tf#/blue_box"]):
rr.set_time("time", sequence=t + 1) # leave it untouched at t==0.
rr.log("point", rr.CoordinateFrame(frame_id))
```
### Transform hierarchy with explicit frames
```python
"""Logs a transform hierarchy using named transform frame relationships."""
import numpy as np
import rerun as rr
rr.init("rerun_example_transform3d_hierarchy_frames", spawn=True)
rr.set_time("sim_time", duration=0)
# Planetary motion is typically in the XY plane.
rr.log("/", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True)
# Setup spheres, all are in the center of their own space:
rr.log(
"sun",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[1, 1, 1],
colors=[255, 200, 10],
fill_mode="solid",
),
rr.CoordinateFrame("sun_frame"),
)
rr.log(
"planet",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.4, 0.4, 0.4],
colors=[40, 80, 200],
fill_mode="solid",
),
rr.CoordinateFrame("planet_frame"),
)
rr.log(
"moon",
rr.Ellipsoids3D(
centers=[0, 0, 0],
half_sizes=[0.15, 0.15, 0.15],
colors=[180, 180, 180],
fill_mode="solid",
),
rr.CoordinateFrame("moon_frame"),
)
# The viewer automatically creates a 3D view at `/`. To connect it to our
# transform hierarchy, we set its coordinate frame to `sun_frame` as well.
# Alternatively, we could also set a blueprint that makes `/sun` the space
# origin.
rr.log("/", rr.CoordinateFrame("sun_frame"))
# Draw fixed paths where the planet & moon move.
d_planet = 6.0
d_moon = 3.0
angles = np.arange(0.0, 1.01, 0.01) * np.pi * 2
circle = np.array(
[np.sin(angles), np.cos(angles), angles * 0.0], dtype=np.float32
).transpose()
rr.log(
"planet_path",
rr.LineStrips3D(circle * d_planet),
rr.CoordinateFrame("sun_frame"),
)
rr.log(
"moon_path",
rr.LineStrips3D(circle * d_moon),
rr.CoordinateFrame("planet_frame"),
)
# Movement via transforms.
for i in range(6 * 120):
time = i / 120.0
rr.set_time("sim_time", duration=time)
r_moon = time * 5.0
r_planet = time * 2.0
rr.log(
"planet_transforms",
rr.Transform3D(
translation=[
np.sin(r_planet) * d_planet,
np.cos(r_planet) * d_planet,
0.0,
],
rotation=rr.RotationAxisAngle(axis=(1, 0, 0), degrees=20),
child_frame="planet_frame",
parent_frame="sun_frame",
),
)
rr.log(
"moon_transforms",
rr.Transform3D(
translation=[np.cos(r_moon) * d_moon, np.sin(r_moon) * d_moon, 0.0],
relation=rr.TransformRelation.ChildFromParent,
child_frame="moon_frame",
parent_frame="planet_frame",
),
)
```
# AnnotationContext
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
The annotation context provides additional information on how to display entities.
Entities can use [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id)s and [`components.KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id)s to provide annotations, and
the labels and colors will be looked up in the appropriate
annotation context. We use the *first* annotation context we find in the
path-hierarchy when searching up through the ancestors of a given entity
path.
See also [`datatypes.ClassDescription`](https://rerun.io/docs/reference/types/datatypes/class_description).
## Fields
### Required
* `context`: [`AnnotationContext`](https://rerun.io/docs/reference/types/components/annotation_context.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `AnnotationContext`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1AnnotationContext.html)
* π [Python API docs for `AnnotationContext`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.AnnotationContext)
* π¦ [Rust API docs for `AnnotationContext`](https://docs.rs/rerun/latest/rerun/archetypes/struct.AnnotationContext.html)
## Examples
### Rectangles
```python
import rerun as rr
rr.init("rerun_example_annotation_context_rects", spawn=True)
# Log an annotation context to assign a label and color to each class
rr.log(
"/",
rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]),
static=True,
)
# Log a batch of 2 rectangles with different `class_ids`
rr.log(
"detections",
rr.Boxes2D(
mins=[[-2, -2], [0, 0]], sizes=[[3, 3], [2, 2]], class_ids=[1, 2]
),
)
```
### Segmentation
```python
"""Log a segmentation image with annotations."""
import numpy as np
import rerun as rr
rr.init("rerun_example_annotation_context_segmentation", spawn=True)
# Create a simple segmentation image
image = np.zeros((200, 300), dtype=np.uint8)
image[50:100, 50:120] = 1
image[100:180, 130:280] = 2
# Log an annotation context to assign a label and color to each class
rr.log(
"segmentation",
rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]),
static=True,
)
rr.log("segmentation/image", rr.SegmentationImage(image))
```
### Connections
```python
"""Log annotation context with connections between keypoints."""
import rerun as rr
from rerun.datatypes import ClassDescription
rr.init("rerun_example_annotation_context_connections", spawn=True)
rr.log(
"/",
rr.AnnotationContext([
ClassDescription(
info=0,
keypoint_annotations=[
(0, "zero", (255, 0, 0)),
(1, "one", (0, 255, 0)),
(2, "two", (0, 0, 255)),
(3, "three", (255, 255, 0)),
],
keypoint_connections=[(0, 2), (1, 2), (2, 3)],
),
]),
static=True,
)
rr.log(
"points",
rr.Points3D(
[
(0, 0, 0),
(50, 0, 20),
(100, 100, 30),
(0, 50, 40),
],
class_ids=[0],
keypoint_ids=[0, 1, 2, 3],
),
)
```
# Mesh3D
A 3D triangle mesh as specified by its per-mesh and per-vertex properties.
See also [`archetypes.Asset3D`](https://rerun.io/docs/reference/types/archetypes/asset3d).
If there are multiple [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) instances logged to the same entity as a mesh,
an instance of the mesh will be drawn for each transform.
For transparency ordering, as well as back face culling (disabled by default),
front faces are assumed to be those with counter clockwise triangle winding order
(this is the same as in the GLTF specification).
## Fields
### Required
* `vertex_positions`: [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md)
### Recommended
* `triangle_indices`: [`TriangleIndices`](https://rerun.io/docs/reference/types/components/triangle_indices.md)
* `vertex_normals`: [`Vector3D`](https://rerun.io/docs/reference/types/components/vector3d.md)
### Optional
* `vertex_colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `vertex_texcoords`: [`Texcoord2D`](https://rerun.io/docs/reference/types/components/texcoord2d.md)
* `albedo_factor`: [`AlbedoFactor`](https://rerun.io/docs/reference/types/components/albedo_factor.md)
* `face_rendering`: [`MeshFaceRendering`](https://rerun.io/docs/reference/types/components/mesh_face_rendering.md)
* `albedo_texture_buffer`: [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* `albedo_texture_format`: [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Mesh3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Mesh3D.html)
* π [Python API docs for `Mesh3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Mesh3D)
* π¦ [Rust API docs for `Mesh3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Mesh3D.html)
## Examples
### Simple indexed 3D mesh
```python
"""Log a simple colored triangle."""
import rerun as rr
rr.init("rerun_example_mesh3d_indexed", spawn=True)
rr.log(
"triangle",
rr.Mesh3D(
vertex_positions=[[0.0, 1.0, 0.0], [1.0, 0.0, 0.0], [0.0, 0.0, 0.0]],
vertex_normals=[0.0, 0.0, 1.0],
vertex_colors=[[0, 0, 255], [0, 255, 0], [255, 0, 0]],
triangle_indices=[2, 1, 0],
),
)
```
### 3D mesh with instancing
```python
"""
Log a simple 3D mesh with several instance pose transforms.
This instantiate the mesh several times and will not
affect its children. This is known as mesh instancing.
"""
import rerun as rr
rr.init("rerun_example_mesh3d_instancing", spawn=True)
rr.set_time("frame", sequence=0)
rr.log(
"shape",
rr.Mesh3D(
vertex_positions=[[1, 1, 1], [-1, -1, 1], [-1, 1, -1], [1, -1, -1]],
triangle_indices=[[0, 2, 1], [0, 3, 1], [0, 3, 2], [1, 3, 2]],
vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255], [255, 255, 0]],
),
)
# This box will not be affected by its parent's instance poses!
rr.log(
"shape/box",
rr.Boxes3D(half_sizes=[[5.0, 5.0, 5.0]]),
)
for i in range(100):
rr.set_time("frame", sequence=i)
rr.log(
"shape",
rr.InstancePoses3D(
translations=[[2, 0, 0], [0, 2, 0], [0, -2, 0], [-2, 0, 0]],
rotation_axis_angles=rr.RotationAxisAngle(
[0, 0, 1], rr.Angle(deg=i * 2)
),
),
)
```
### Update specific parts of a 3D mesh over time
```python
"""Log a colored triangle, then update its vertices' positions each frame."""
import numpy as np
import rerun as rr
rr.init("rerun_example_mesh3d_partial_updates", spawn=True)
vertex_positions = np.array(
[[-1.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], dtype=np.float32
)
# Log the initial state of our triangle
rr.set_time("frame", sequence=0)
rr.log(
"triangle",
rr.Mesh3D(
vertex_positions=vertex_positions,
vertex_normals=[0.0, 0.0, 1.0],
vertex_colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255]],
),
)
# Only update its vertices' positions each frame:
for i in range(1, 300):
factor = np.abs(np.sin(i * 0.04))
rr.set_time("frame", sequence=i)
rr.log(
"triangle",
rr.Mesh3D.from_fields(vertex_positions=vertex_positions * factor),
)
```
# Arrows3D
3D arrows with optional colors, radii, labels, etc.
## Fields
### Required
* `vectors`: [`Vector3D`](https://rerun.io/docs/reference/types/components/vector3d.md)
### Recommended
* `origins`: [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md)
### Optional
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Arrows3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Arrows3D.html)
* π [Python API docs for `Arrows3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Arrows3D)
* π¦ [Rust API docs for `Arrows3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Arrows3D.html)
## Example
### Simple batch of 3D arrows
```python
"""Log a batch of 3D arrows."""
from math import tau
import numpy as np
import rerun as rr
rr.init("rerun_example_arrow3d", spawn=True)
lengths = np.log2(np.arange(0, 100) + 1)
angles = np.arange(start=0, stop=tau, step=tau * 0.01)
origins = np.zeros((100, 3))
vectors = np.column_stack([
np.sin(angles) * lengths,
np.zeros(100),
np.cos(angles) * lengths,
])
colors = [[1.0 - c, c, 0.5, 0.5] for c in angles / tau]
rr.log("arrows", rr.Arrows3D(origins=origins, vectors=vectors, colors=colors))
```
# GeoLineStrings
Geospatial line strings with positions expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees), and optional colors and radii.
Also known as "line strips" or "polylines".
## Fields
### Required
* `line_strings`: [`GeoLineString`](https://rerun.io/docs/reference/types/components/geo_line_string.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
## Can be shown in
* [MapView](https://rerun.io/docs/reference/types/views/map_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `GeoLineStrings`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1GeoLineStrings.html)
* π [Python API docs for `GeoLineStrings`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.GeoLineStrings)
* π¦ [Rust API docs for `GeoLineStrings`](https://docs.rs/rerun/latest/rerun/archetypes/struct.GeoLineStrings.html)
## Example
### Log a geospatial line string
```python
"""Log a simple geospatial line string."""
import rerun as rr
rr.init("rerun_example_geo_line_strings", spawn=True)
rr.log(
"colorado",
rr.GeoLineStrings(
lat_lon=[
[41.0000, -109.0452],
[41.0000, -102.0415],
[36.9931, -102.0415],
[36.9931, -109.0452],
[41.0000, -109.0452],
],
radii=rr.Radius.ui_points(2.0),
colors=[0, 0, 255],
),
)
```
# ViewCoordinates
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
How we interpret the coordinate system of an entity/space.
For instance: What is "up"? What does the Z axis mean?
The three coordinates are always ordered as [x, y, z].
For example [Right, Down, Forward] means that the X axis points to the right, the Y axis points
down, and the Z axis points forward.
Make sure that this archetype is logged at or above the origin entity path of your 3D views.
β [Rerun does not yet support left-handed coordinate systems](https://github.com/rerun-io/rerun/issues/5032).
## Fields
### Required
* `xyz`: [`ViewCoordinates`](https://rerun.io/docs/reference/types/components/view_coordinates.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `ViewCoordinates`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1ViewCoordinates.html)
* π [Python API docs for `ViewCoordinates`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.ViewCoordinates)
* π¦ [Rust API docs for `ViewCoordinates`](https://docs.rs/rerun/latest/rerun/archetypes/struct.ViewCoordinates.html)
## Example
### View coordinates for adjusting the eye camera
```python
"""Change the view coordinates for the scene."""
import rerun as rr
rr.init("rerun_example_view_coordinates", spawn=True)
rr.log(
"world", rr.ViewCoordinates.RIGHT_HAND_Z_UP, static=True
) # Set an up-axis
rr.log(
"world/xyz",
rr.Arrows3D(
vectors=[[1, 0, 0], [0, 1, 0], [0, 0, 1]],
colors=[[255, 0, 0], [0, 255, 0], [0, 0, 255]],
),
)
```
# Arrows2D
2D arrows with optional colors, radii, labels, etc.
## Fields
### Required
* `vectors`: [`Vector2D`](https://rerun.io/docs/reference/types/components/vector2d.md)
### Recommended
* `origins`: [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
### Optional
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Arrows2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Arrows2D.html)
* π [Python API docs for `Arrows2D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Arrows2D)
* π¦ [Rust API docs for `Arrows2D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Arrows2D.html)
## Example
### Simple batch of 2D arrows
```python
"""Log a batch of 2D arrows."""
import rerun as rr
rr.init("rerun_example_arrow2d", spawn=True)
rr.log(
"arrows",
rr.Arrows2D(
origins=[[0.25, 0.0], [0.25, 0.0], [-0.1, -0.1]],
vectors=[[1.0, 0.0], [0.0, -1.0], [-0.7, 0.7]],
colors=[[255, 0, 0], [0, 255, 0], [127, 0, 255]],
labels=["right", "up", "left-down"],
radii=0.025,
),
)
```
# Pinhole
Camera perspective projection (a.k.a. intrinsics).
If [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d) is logged for the same child/parent relationship (e.g. for the camera extrinsics), it takes precedence over [`archetypes.Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole).
If you use named transform frames via the `child_frame` and `parent_frame` fields, you don't have to use [`archetypes.CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame)
as it is the case with other visualizations: for any entity with an [`archetypes.Pinhole`](https://rerun.io/docs/reference/types/archetypes/pinhole) the viewer will always visualize it
directly without needing a [`archetypes.CoordinateFrame`](https://rerun.io/docs/reference/types/archetypes/coordinate_frame) to refer to the pinhole's child/parent frame.
## Fields
### Required
* `image_from_camera`: [`PinholeProjection`](https://rerun.io/docs/reference/types/components/pinhole_projection.md)
### Recommended
* `resolution`: [`Resolution`](https://rerun.io/docs/reference/types/components/resolution.md)
### Optional
* `camera_xyz`: [`ViewCoordinates`](https://rerun.io/docs/reference/types/components/view_coordinates.md)
* `child_frame`: [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
* `parent_frame`: [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
* `image_plane_distance`: [`ImagePlaneDistance`](https://rerun.io/docs/reference/types/components/image_plane_distance.md)
* `color`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `line_width`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Pinhole`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Pinhole.html)
* π [Python API docs for `Pinhole`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Pinhole)
* π¦ [Rust API docs for `Pinhole`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Pinhole.html)
## Examples
### Simple pinhole camera
```python
"""Log a pinhole and a random image."""
import numpy as np
import rerun as rr
rr.init("rerun_example_pinhole", spawn=True)
rng = np.random.default_rng(12345)
image = rng.uniform(0, 255, size=[3, 3, 3])
rr.log("world/image", rr.Pinhole(focal_length=3, width=3, height=3))
rr.log("world/image", rr.Image(image))
```
### Perspective pinhole camera
```python
"""Logs a point cloud and a perspective camera looking at it."""
import rerun as rr
rr.init("rerun_example_pinhole_perspective", spawn=True)
rr.log(
"world/cam",
rr.Pinhole(
fov_y=0.7853982,
aspect_ratio=1.7777778,
camera_xyz=rr.ViewCoordinates.RUB,
image_plane_distance=0.1,
color=[255, 128, 0],
line_width=0.003,
),
)
rr.log(
"world/points",
rr.Points3D(
[(0.0, 0.0, -0.5), (0.1, 0.1, -0.5), (-0.1, -0.1, -0.5)], radii=0.025
),
)
```
### Projection setup with blueprints
```python
"""Demonstrates pinhole camera projections with Rerun blueprints."""
import numpy as np
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_pinhole_projections", spawn=True)
img_height, img_width = 12, 16
# Create a 3D scene with a camera and an image.
rr.log(
"world/box",
rr.Boxes3D(centers=[0, 0, 0], half_sizes=[1, 1, 1], colors=[255, 0, 0]),
)
rr.log(
"world/points",
rr.Points3D(
positions=[(1, 0, 0), (-1, 0, 0), (0, 1, 0), (0, -1, 0), (0, 0, 1)],
colors=[
(255, 0, 0),
(0, 255, 0),
(0, 0, 255),
(255, 255, 0),
(255, 0, 255),
],
radii=0.1,
),
)
rr.log(
"camera",
rr.Transform3D(translation=[0, 3, 0]),
rr.Pinhole(
width=img_width,
height=img_height,
focal_length=10,
camera_xyz=rr.ViewCoordinates.LEFT_HAND_Z_UP,
),
)
# Create a simple test image.
checkerboard = np.zeros((img_height, img_width, 1), dtype=np.uint8)
checkerboard[
(np.arange(img_height)[:, None] + np.arange(img_width)) % 2 == 0
] = 255
rr.log("camera/image", rr.Image(checkerboard))
# Use a blueprint to show both 3D and 2D views side by side.
blueprint = rrb.Blueprint(
rrb.Horizontal(
# 3D view showing the scene and camera
rrb.Spatial3DView(
origin="world",
name="3D Scene",
contents=["/**"],
overrides={
# Adjust visual size of camera frustum in 3D view for
# better visibility.
"camera": rr.Pinhole.from_fields(image_plane_distance=1.0)
},
),
# 2D projection from angled camera
rrb.Spatial2DView(
# Make sure that the origin is at the camera's path.
origin="camera",
name="Camera",
contents=["/**"], # Add everything, so 3D objects get projected.
),
)
)
rr.send_blueprint(blueprint)
```
# Points3D
A 3D point cloud with positions and optional colors, radii, labels, etc.
If there are multiple instance poses, the entire point cloud will be repeated for each of the poses.
## Fields
### Required
* `positions`: [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `point_shading`: [`PointShading`](https://rerun.io/docs/reference/types/components/point_shading.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
* `keypoint_ids`: [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Points3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Points3D.html)
* π [Python API docs for `Points3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Points3D)
* π¦ [Rust API docs for `Points3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Points3D.html)
## Examples
### Simple 3D points
```python
"""Log some very simple points."""
import rerun as rr
rr.init("rerun_example_points3d", spawn=True)
rr.log("points", rr.Points3D([[0, 0, 0], [1, 1, 1]]))
```
### Randomly distributed 3D points with varying color and radius
```python
"""Log some random points with color and radii."""
from numpy.random import default_rng
import rerun as rr
rr.init("rerun_example_points3d_random", spawn=True)
rng = default_rng(12345)
positions = rng.uniform(-5, 5, size=[10, 3])
colors = rng.uniform(0, 255, size=[10, 3])
radii = rng.uniform(0, 1, size=[10])
rr.log("random", rr.Points3D(positions, colors=colors, radii=radii))
```
### Log points with radii given in UI points
```python
"""Log some points with ui points & scene unit radii."""
import rerun as rr
rr.init("rerun_example_points3d_ui_radius", spawn=True)
# Two blue points with scene unit radii of 0.1 and 0.3.
rr.log(
"scene_units",
rr.Points3D(
[[0, 1, 0], [1, 1, 1]],
# By default, radii are interpreted as world-space units.
radii=[0.1, 0.3],
colors=[0, 0, 255],
),
)
# Two red points with ui point radii of 40 and 60.
# UI points are independent of zooming in Views, but are sensitive to the
# application UI scaling.
# For 100% ui scaling, UI points are equal to pixels.
rr.log(
"ui_points",
rr.Points3D(
[[0, 0, 0], [1, 0, 1]],
# rr.Radius.ui_points produces radii that the viewer interprets
# as given in ui points.
radii=rr.Radius.ui_points([40.0, 60.0]),
colors=[255, 0, 0],
),
)
```
### Update a point cloud over time
```python
"""
Update a point cloud over time.
See also the `points3d_column_updates` example, which achieves the same
thing in a single operation.
"""
import numpy as np
import rerun as rr
rr.init("rerun_example_points3d_row_updates", spawn=True)
# Prepare a point cloud that evolves over 5 timesteps, changing the
# number of points in the process.
times = np.arange(10, 15, 1.0)
# fmt: off
positions = [
[[1.0, 0.0, 1.0], [0.5, 0.5, 2.0]],
[[1.5, -0.5, 1.5], [1.0, 1.0, 2.5], [-0.5, 1.5, 1.0], [-1.5, 0.0, 2.0]],
[[2.0, 0.0, 2.0], [1.5, -1.5, 3.0], [0.0, -2.0, 2.5], [1.0, -1.0, 3.5]],
[[-2.0, 0.0, 2.0], [-1.5, 1.5, 3.0], [-1.0, 1.0, 3.5]],
[[1.0, -1.0, 1.0], [2.0, -2.0, 2.0], [3.0, -1.0, 3.0], [2.0, 0.0, 4.0]],
]
# fmt: on
# At each timestep, all points in the cloud share the same but changing
# color and radius.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
radii = [0.05, 0.01, 0.2, 0.1, 0.3]
for i in range(5):
rr.set_time("time", duration=10 + i)
rr.log(
"points", rr.Points3D(positions[i], colors=colors[i], radii=radii[i])
)
```
### Update a point cloud over time, in a single operation
```python
"""
Update a point cloud over time, in a single operation.
This is semantically equivalent to the `points3d_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_points3d_column_updates", spawn=True)
# Prepare a point cloud that evolves over 5 timesteps, changing the
# number of points in the process.
times = np.arange(10, 15, 1.0)
# fmt: off
positions = [
[1.0, 0.0, 1.0], [0.5, 0.5, 2.0],
[1.5, -0.5, 1.5], [1.0, 1.0, 2.5], [-0.5, 1.5, 1.0], [-1.5, 0.0, 2.0],
[2.0, 0.0, 2.0], [1.5, -1.5, 3.0], [0.0, -2.0, 2.5], [1.0, -1.0, 3.5],
[-2.0, 0.0, 2.0], [-1.5, 1.5, 3.0], [-1.0, 1.0, 3.5],
[1.0, -1.0, 1.0], [2.0, -2.0, 2.0], [3.0, -1.0, 3.0], [2.0, 0.0, 4.0],
]
# fmt: on
# At each timestep, all points in the cloud share the same but changing
# color and radius.
colors = [0xFF0000FF, 0x00FF00FF, 0x0000FFFF, 0xFFFF00FF, 0x00FFFFFF]
radii = [0.05, 0.01, 0.2, 0.1, 0.3]
rr.send_columns(
"points",
indexes=[rr.TimeColumn("time", duration=times)],
columns=[
*rr.Points3D.columns(positions=positions).partition(
lengths=[2, 4, 4, 3, 4]
),
*rr.Points3D.columns(colors=colors, radii=radii),
],
)
```
### Update specific properties of a point cloud over time
```python
"""Update specific properties of a point cloud over time."""
import rerun as rr
rr.init("rerun_example_points3d_partial_updates", spawn=True)
positions = [[i, 0, 0] for i in range(10)]
rr.set_time("frame", sequence=0)
rr.log("points", rr.Points3D(positions))
for i in range(10):
colors = [[20, 200, 20] if n < i else [200, 20, 20] for n in range(10)]
radii = [0.6 if n < i else 0.2 for n in range(10)]
# Update only the colors and radii, leaving everything else as-is.
rr.set_time("frame", sequence=i)
rr.log("points", rr.Points3D.from_fields(radii=radii, colors=colors))
# Update the positions and radii, and clear everything else in the process.
rr.set_time("frame", sequence=20)
rr.log(
"points",
rr.Points3D.from_fields(clear_unset=True, positions=positions, radii=0.3),
)
```
# McapSchema
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A schema definition that describes the structure of messages in an MCAP file.
Schemas define the data types and field structures used by messages in MCAP channels.
They provide the blueprint for interpreting message payloads, specifying field names,
types, and organization. Each schema is referenced by channels to indicate how their
messages should be decoded and understood.
See also [`archetypes.McapChannel`](https://rerun.io/docs/reference/types/archetypes/mcap_channel) for channels that reference these schemas,
[`archetypes.McapMessage`](https://rerun.io/docs/reference/types/archetypes/mcap_message) for the messages that conform to these schemas, and the
[MCAP specification](https://mcap.dev/) for complete format details.
## Fields
### Required
* `id`: [`SchemaId`](https://rerun.io/docs/reference/types/components/schema_id.md)
* `name`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `encoding`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `data`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
## Can be shown in
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `McapSchema`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1McapSchema.html)
* π [Python API docs for `McapSchema`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.McapSchema)
* π¦ [Rust API docs for `McapSchema`](https://docs.rs/rerun/latest/rerun/archetypes/struct.McapSchema.html)
## Example
### Simple MCAP schema
```python
"""Log a simple MCAP schema definition."""
import rerun as rr
rr.init("rerun_example_mcap_schema", spawn=True)
# Example ROS2 message definition for a simple Point message
point_schema = """float64 x
float64 y
float64 z"""
rr.log(
"mcap/schemas/geometry_point",
rr.McapSchema(
id=42,
name="geometry_msgs/msg/Point",
encoding="ros2msg",
data=point_schema.encode("utf-8"),
),
)
```
# Boxes3D
3D boxes with half-extents and optional center, rotations, colors etc.
If there's more instance poses than half sizes, the last box's orientation will be repeated for the remaining poses.
Orienting and placing boxes forms a separate transform that is applied prior to [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) and [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
## Fields
### Required
* `half_sizes`: [`HalfSize3D`](https://rerun.io/docs/reference/types/components/half_size3d.md)
### Recommended
* `centers`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `rotation_axis_angles`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternions`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `fill_mode`: [`FillMode`](https://rerun.io/docs/reference/types/components/fill_mode.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Boxes3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Boxes3D.html)
* π [Python API docs for `Boxes3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Boxes3D)
* π¦ [Rust API docs for `Boxes3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Boxes3D.html)
## Examples
### Simple 3D boxes
```python
"""Log a single 3D Box."""
import rerun as rr
rr.init("rerun_example_box3d", spawn=True)
rr.log("simple", rr.Boxes3D(half_sizes=[2.0, 2.0, 1.0]))
```
### Batch of 3D boxes
```python
"""Log a batch of oriented bounding boxes."""
import rerun as rr
rr.init("rerun_example_box3d_batch", spawn=True)
rr.log(
"batch",
rr.Boxes3D(
centers=[[2, 0, 0], [-2, 0, 0], [0, 0, 2]],
half_sizes=[[2.0, 2.0, 1.0], [1.0, 1.0, 0.5], [2.0, 0.5, 1.0]],
quaternions=[
rr.Quaternion.identity(),
rr.Quaternion(
xyzw=[0.0, 0.0, 0.382683, 0.923880]
), # 45 degrees around Z
],
radii=0.025,
colors=[(255, 0, 0), (0, 255, 0), (0, 0, 255)],
fill_mode="solid",
labels=["red", "green", "blue"],
),
)
```
# GraphNodes
A list of nodes in a graph with optional labels, colors, etc.
## Fields
### Required
* `node_ids`: [`GraphNode`](https://rerun.io/docs/reference/types/components/graph_node.md)
### Optional
* `positions`: [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
## Can be shown in
* [GraphView](https://rerun.io/docs/reference/types/views/graph_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `GraphNodes`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1GraphNodes.html)
* π [Python API docs for `GraphNodes`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.GraphNodes)
* π¦ [Rust API docs for `GraphNodes`](https://docs.rs/rerun/latest/rerun/archetypes/struct.GraphNodes.html)
## Examples
### Simple undirected graph
```python
"""Log a simple undirected graph."""
import rerun as rr
rr.init("rerun_example_graph_undirected", spawn=True)
rr.log(
"simple",
rr.GraphNodes(
node_ids=["a", "b", "c"],
positions=[(0.0, 100.0), (-100.0, 0.0), (100.0, 0.0)],
labels=["A", "B", "C"],
),
rr.GraphEdges(
edges=[("a", "b"), ("b", "c"), ("c", "a")],
# Optional: graphs are undirected by default.
graph_type="undirected",
),
)
```
### Simple directed graph
```python
"""Log a simple directed graph."""
import rerun as rr
rr.init("rerun_example_graph_directed", spawn=True)
rr.log(
"simple",
rr.GraphNodes(
node_ids=["a", "b", "c"],
positions=[(0.0, 100.0), (-100.0, 0.0), (100.0, 0.0)],
labels=["A", "B", "C"],
),
rr.GraphEdges(
edges=[("a", "b"), ("b", "c"), ("c", "a")], graph_type="directed"
),
)
```
# GridMap
A 2D grid map stored as raster data in an image buffer, with a cell size in scene units and pose.
This archetype is intended for robotics applications like occupancy maps or navigation costmaps.
## Fields
### Required
* `data`: [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* `format`: [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
* `cell_size`: [`CellSize`](https://rerun.io/docs/reference/types/components/cell_size.md)
### Optional
* `translation`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `rotation_axis_angle`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternion`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `colormap`: [`Colormap`](https://rerun.io/docs/reference/types/components/colormap.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `GridMap`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1GridMap.html)
* π [Python API docs for `GridMap`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.GridMap)
* π¦ [Rust API docs for `GridMap`](https://docs.rs/rerun/latest/rerun/archetypes/struct.GridMap.html)
## Examples
### Simple occupancy grid map
```python
"""Log a simple occupancy grid map."""
import numpy as np
import rerun as rr
width, height = 64, 64
cell_size = 0.1
# Create a synthetic image with ROS `nav_msgs/OccupancyGrid` cell value
# conventions: -1 (255) unknown, 0 free, 100 occupied.
grid = np.full((height, width), -1, dtype=np.int8)
grid[8:56, 8:56] = 0
grid[20:44, 20:44] = 100
rr.init("rerun_example_grid_map", spawn=True)
rr.log(
"world/map",
rr.GridMap(
data=grid.tobytes(),
format=rr.components.ImageFormat(
width=width,
height=height,
color_model="L",
channel_datatype="U8",
),
cell_size=cell_size,
translation=[
-(width * cell_size) / 2.0,
-(height * cell_size) / 2.0,
0.0,
],
colormap=rr.components.Colormap.RvizMap,
),
)
```
### Log a grid map at a specific pose
```python
"""Shows how to log a GridMap at a specific pose."""
import math
from pathlib import Path
from PIL import Image as PILImage
import rerun as rr
import rerun.blueprint as rrb
rr.init("rerun_example_grid_map_pose", spawn=True)
# Log the transform for the map origin.
# Here we use ROS TF-style parent & child frame names.
rr.log(
"/tf",
rr.Transform3D(
translation=[1.0, 2.0, 0.0],
rotation_axis_angle=rr.components.RotationAxisAngle(
[0, 0, 1], -math.pi / 3
),
parent_frame="world",
child_frame="map",
),
static=True,
)
# We use a dummy image for the map in this example.
image = PILImage.open(Path(__file__).parent / "ferris.png").convert("RGBA")
# Log the grid map at the map origin.
rr.log(
"demo_map",
rr.CoordinateFrame("map"),
rr.GridMap(
data=image.tobytes(),
format=rr.components.ImageFormat(
width=image.size[0],
height=image.size[1],
color_model="RGBA",
channel_datatype="U8",
),
opacity=0.5,
# The size of a pixel in scene units.
cell_size=0.01,
# Specify the pose of the lower-left image corner relative to the
# map frame, in scene units.
translation=[1.1, -1.6, 0.0],
rotation_axis_angle=rr.components.RotationAxisAngle(
[0, 0, 1], math.pi / 4.0
),
),
)
# Show transform axes with frame names.
rr.send_blueprint(
rrb.Spatial3DView(
origin="/",
overrides={
"/tf": [rr.TransformAxes3D(axis_length=0.5, show_frame=True)],
},
)
)
```
# EncodedDepthImage
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A depth image encoded with a codec (e.g. RVL or PNG).
Rerun also supports uncompressed depth images with the [`archetypes.DepthImage`](https://rerun.io/docs/reference/types/archetypes/depth_image).
## Fields
### Required
* `blob`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
### Recommended
* `media_type`: [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
* `meter`: [`DepthMeter`](https://rerun.io/docs/reference/types/components/depth_meter.md)
### Optional
* `colormap`: [`Colormap`](https://rerun.io/docs/reference/types/components/colormap.md)
* `depth_range`: [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md)
* `point_fill_ratio`: [`FillRatio`](https://rerun.io/docs/reference/types/components/fill_ratio.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `magnification_filter`: [`MagnificationFilter`](https://rerun.io/docs/reference/types/components/magnification_filter.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `EncodedDepthImage`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1EncodedDepthImage.html)
* π [Python API docs for `EncodedDepthImage`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.EncodedDepthImage)
* π¦ [Rust API docs for `EncodedDepthImage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.EncodedDepthImage.html)
## Example
### Encoded depth image
```python
"""Log an encoded depth image stored as a 16-bit PNG or RVL file."""
import sys
from pathlib import Path
import rerun as rr
if len(sys.argv) < 2:
print(
f"Usage: {sys.argv[0]} ", file=sys.stderr
)
sys.exit(1)
depth_path = Path(sys.argv[1])
rr.init("rerun_example_encoded_depth_image", spawn=True)
depth_png = depth_path.read_bytes()
if depth_path.suffix.lower() == ".png":
media_type = rr.components.MediaType.PNG
else:
media_type = rr.components.MediaType.RVL
rr.log(
"depth/encoded",
rr.EncodedDepthImage(
blob=depth_png,
media_type=media_type,
meter=0.001,
),
)
```
# Ellipses2D
2D ellipses with half-extents (semi-axes) and optional center, colors etc.
The half-sizes specify the lengths of the ellipse's two axes along the local x and y directions.
If both half-sizes are equal, the ellipse is a circle.
## Fields
### Required
* `half_sizes`: [`HalfSize2D`](https://rerun.io/docs/reference/types/components/half_size2d.md)
### Recommended
* `centers`: [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `line_radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Ellipses2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Ellipses2D.html)
* π [Python API docs for `Ellipses2D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Ellipses2D)
* π¦ [Rust API docs for `Ellipses2D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Ellipses2D.html)
## Examples
### Simple 2D ellipses
```python
"""Log a simple 2D ellipse."""
import rerun as rr
rr.init("rerun_example_ellipses2d", spawn=True)
rr.log("simple", rr.Ellipses2D(half_sizes=[(2.0, 1.0)], centers=[(0.0, 0.0)]))
```
### Batch of 2D ellipses
```python
"""Log a batch of 2D ellipses."""
import rerun as rr
rr.init("rerun_example_ellipses2d_batch", spawn=True)
rr.log(
"batch",
rr.Ellipses2D(
centers=[(-2.0, 0.0), (0.0, 0.0), (2.5, 0.0)],
half_sizes=[(1.5, 0.75), (0.5, 0.5), (0.75, 1.5)],
line_radii=[0.025, 0.05, 0.025],
colors=[(255, 0, 0), (0, 255, 0), (0, 0, 255)],
labels=["wide", "circle", "tall"],
),
)
```
# GeoPoints
Geospatial points with positions expressed in [EPSG:4326](https://epsg.io/4326) latitude and longitude (North/East-positive degrees), and optional colors and radii.
## Fields
### Required
* `positions`: [`LatLon`](https://rerun.io/docs/reference/types/components/lat_lon.md)
### Recommended
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [MapView](https://rerun.io/docs/reference/types/views/map_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `GeoPoints`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1GeoPoints.html)
* π [Python API docs for `GeoPoints`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.GeoPoints)
* π¦ [Rust API docs for `GeoPoints`](https://docs.rs/rerun/latest/rerun/archetypes/struct.GeoPoints.html)
## Example
### Log a geospatial point
```python
"""Log some very simple geospatial point."""
import rerun as rr
rr.init("rerun_example_geo_points", spawn=True)
rr.log(
"rerun_hq",
rr.GeoPoints(
lat_lon=[59.319221, 18.075631],
radii=rr.Radius.ui_points(10.0),
colors=[255, 0, 0],
),
)
```
# TextDocument
A text element intended to be displayed in its own text box.
Supports raw text and markdown.
## Fields
### Required
* `text`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
### Optional
* `media_type`: [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
## Can be shown in
* [TextDocumentView](https://rerun.io/docs/reference/types/views/text_document_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `TextDocument`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1TextDocument.html)
* π [Python API docs for `TextDocument`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.TextDocument)
* π¦ [Rust API docs for `TextDocument`](https://docs.rs/rerun/latest/rerun/archetypes/struct.TextDocument.html)
## Example
### Markdown text document
```python
"""Log a `TextDocument`."""
import rerun as rr
rr.init("rerun_example_text_document", spawn=True)
rr.log("text_document", rr.TextDocument("Hello, TextDocument!"))
rr.log(
"markdown",
rr.TextDocument(
'''
# Hello Markdown!
[Click here to see the raw text](recording://markdown:Text).
Basic formatting:
| **Feature** | **Alternative** |
| ----------------- | --------------- |
| Plain | |
| *italics* | _italics_ |
| **bold** | __bold__ |
| ~~strikethrough~~ | |
| `inline code` | |
----------------------------------
## Support
- [x] [Commonmark](https://commonmark.org/help/) support
- [x] GitHub-style strikethrough, tables, and checkboxes
- Basic syntax highlighting for:
- [x] C and C++
- [x] Python
- [x] Rust
- [ ] Other languages
## Links
You can link to [an entity](recording://markdown),
a [specific instance of an entity](recording://markdown[#0]),
or a [specific component](recording://markdown:Text).
Of course you can also have [normal https links](https://github.com/rerun-io/rerun), e.g. .
## Image
'''.strip(),
media_type=rr.MediaType.MARKDOWN,
),
)
```
# AssetVideo
A video binary.
Only MP4 containers are currently supported.
See for codec support and more general information.
In order to display a video, you also need to log a [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference) for each frame.
## Fields
### Required
* `blob`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
### Recommended
* `media_type`: [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `AssetVideo`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1AssetVideo.html)
* π [Python API docs for `AssetVideo`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.AssetVideo)
* π¦ [Rust API docs for `AssetVideo`](https://docs.rs/rerun/latest/rerun/archetypes/struct.AssetVideo.html)
## Examples
### Video with automatically determined frames
```python
"""Log a video asset using automatically determined frame references."""
import sys
import rerun as rr
if len(sys.argv) < 2:
# TODO(#7354): Only mp4 is supported for now.
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset_video_auto_frames", spawn=True)
# Log video asset which is referred to by frame references.
video_asset = rr.AssetVideo(path=sys.argv[1])
rr.log("video", video_asset, static=True)
# Send automatically determined video frame timestamps.
frame_timestamps_ns = video_asset.read_frame_timestamps_nanos()
rr.send_columns(
"video",
# Note timeline values don't have to be the same as the video timestamps.
indexes=[rr.TimeColumn("video_time", duration=1e-9 * frame_timestamps_ns)],
columns=rr.VideoFrameReference.columns_nanos(frame_timestamps_ns),
)
```
### Demonstrates manual use of video frame references
```python
"""Manual use of individual video frame references."""
import sys
import rerun as rr
import rerun.blueprint as rrb
if len(sys.argv) < 2:
# TODO(#7354): Only mp4 is supported for now.
print(f"Usage: {sys.argv[0]} ")
sys.exit(1)
rr.init("rerun_example_asset_video_manual_frames", spawn=True)
# Log video asset which is referred to by frame references.
rr.log("video_asset", rr.AssetVideo(path=sys.argv[1]), static=True)
# Create two entities, showing the same video frozen at different times.
rr.log(
"frame_1s",
rr.VideoFrameReference(seconds=1.0, video_reference="video_asset"),
)
rr.log(
"frame_2s",
rr.VideoFrameReference(seconds=2.0, video_reference="video_asset"),
)
# Send blueprint that shows two 2D views next to each other.
rr.send_blueprint(
rrb.Horizontal(
rrb.Spatial2DView(origin="frame_1s"),
rrb.Spatial2DView(origin="frame_2s"),
)
)
```
# Ellipsoids3D
3D ellipsoids or spheres.
This archetype is for ellipsoids or spheres whose size is a key part of the data
(e.g. a bounding sphere).
For points whose radii are for the sake of visualization, use [`archetypes.Points3D`](https://rerun.io/docs/reference/types/archetypes/points3d) instead.
If there's more instance poses than half sizes, the last ellipsoid/sphere's orientation will be repeated for the remaining poses.
Orienting and placing ellipsoids/spheres forms a separate transform that is applied prior to [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) and [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
## Fields
### Required
* `half_sizes`: [`HalfSize3D`](https://rerun.io/docs/reference/types/components/half_size3d.md)
### Recommended
* `centers`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `rotation_axis_angles`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternions`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `line_radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `fill_mode`: [`FillMode`](https://rerun.io/docs/reference/types/components/fill_mode.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Ellipsoids3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Ellipsoids3D.html)
* π [Python API docs for `Ellipsoids3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Ellipsoids3D)
* π¦ [Rust API docs for `Ellipsoids3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Ellipsoids3D.html)
## Example
### Covariance ellipsoid
```python
"""Log random points and the corresponding covariance ellipsoid."""
import numpy as np
import rerun as rr
rr.init("rerun_example_ellipsoid_simple", spawn=True)
center = np.array([0, 0, 0])
sigmas = np.array([5, 3, 1])
points = np.random.randn(50_000, 3) * sigmas.reshape(1, -1)
rr.log("points", rr.Points3D(points, radii=0.02, colors=[188, 77, 185]))
rr.log(
"ellipsoid",
rr.Ellipsoids3D(
centers=[center, center],
half_sizes=[sigmas, 3 * sigmas],
colors=[[255, 255, 0], [64, 64, 0]],
),
)
```
# Tensor
An N-dimensional array of numbers.
## Fields
### Required
* `data`: [`TensorData`](https://rerun.io/docs/reference/types/components/tensor_data.md)
### Optional
* `value_range`: [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md)
## Can be shown in
* [TensorView](https://rerun.io/docs/reference/types/views/tensor_view.md)
* [BarChartView](https://rerun.io/docs/reference/types/views/bar_chart_view.md) (for 1D tensors)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Tensor`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Tensor.html)
* π [Python API docs for `Tensor`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Tensor)
* π¦ [Rust API docs for `Tensor`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Tensor.html)
## Example
### Simple tensor
```python
"""Create and log a tensor."""
import numpy as np
import rerun as rr
tensor = np.random.randint(
0, 256, (8, 6, 3, 5), dtype=np.uint8
) # 4-dimensional tensor
rr.init("rerun_example_tensor", spawn=True)
# Log the tensor, assigning names to each dimension
rr.log(
"tensor",
rr.Tensor(tensor, dim_names=("width", "height", "channel", "batch")),
)
```
# EncodedImage
An image encoded as e.g. a JPEG or PNG.
Rerun also supports uncompressed images with the [`archetypes.Image`](https://rerun.io/docs/reference/types/archetypes/image).
For images that refer to video frames see [`archetypes.VideoFrameReference`](https://rerun.io/docs/reference/types/archetypes/video_frame_reference).
## Fields
### Required
* `blob`: [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
### Recommended
* `media_type`: [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
### Optional
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* `magnification_filter`: [`MagnificationFilter`](https://rerun.io/docs/reference/types/components/magnification_filter.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `EncodedImage`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1EncodedImage.html)
* π [Python API docs for `EncodedImage`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.EncodedImage)
* π¦ [Rust API docs for `EncodedImage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.EncodedImage.html)
## Example
### Encoded image
```python
"""Create and log an image."""
from pathlib import Path
import rerun as rr
image_file_path = Path(__file__).parent / "ferris.png"
rr.init("rerun_example_encoded_image", spawn=True)
rr.log("image", rr.EncodedImage(path=image_file_path))
```
# Clear
Empties all the components of an entity.
The presence of a clear means that a latest-at query of components at a given path(s)
will not return any components that were logged at those paths before the clear.
Any logged components after the clear are unaffected by the clear.
This implies that a range query that includes time points that are before the clear,
still returns all components at the given path(s).
Meaning that in practice clears are ineffective when making use of visible time ranges.
Scalar plots are an exception: they track clears and use them to represent holes in the
data (i.e. discontinuous lines).
## Fields
### Required
* `is_recursive`: [`ClearIsRecursive`](https://rerun.io/docs/reference/types/components/clear_is_recursive.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [TimeSeriesView](https://rerun.io/docs/reference/types/views/time_series_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Clear`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Clear.html)
* π [Python API docs for `Clear`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Clear)
* π¦ [Rust API docs for `Clear`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Clear.html)
## Examples
### Flat
```python
"""Log and then clear data."""
import rerun as rr
rr.init("rerun_example_clear", spawn=True)
vectors = [(1.0, 0.0, 0.0), (0.0, -1.0, 0.0), (-1.0, 0.0, 0.0), (0.0, 1.0, 0.0)]
origins = [
(-0.5, 0.5, 0.0),
(0.5, 0.5, 0.0),
(0.5, -0.5, 0.0),
(-0.5, -0.5, 0.0),
]
colors = [(200, 0, 0), (0, 200, 0), (0, 0, 200), (200, 0, 200)]
# Log a handful of arrows.
for i, (vector, origin, color) in enumerate(
zip(vectors, origins, colors, strict=False)
):
rr.log(
f"arrows/{i}", rr.Arrows3D(vectors=vector, origins=origin, colors=color)
)
# Now clear them, one by one on each tick.
for i in range(len(vectors)):
rr.log(f"arrows/{i}", rr.Clear(recursive=False)) # or `rr.Clear.flat()`
```
### Recursive
```python
"""Log and then clear data recursively."""
import rerun as rr
rr.init("rerun_example_clear_recursive", spawn=True)
vectors = [(1.0, 0.0, 0.0), (0.0, -1.0, 0.0), (-1.0, 0.0, 0.0), (0.0, 1.0, 0.0)]
origins = [
(-0.5, 0.5, 0.0),
(0.5, 0.5, 0.0),
(0.5, -0.5, 0.0),
(-0.5, -0.5, 0.0),
]
colors = [(200, 0, 0), (0, 200, 0), (0, 0, 200), (200, 0, 200)]
# Log a handful of arrows.
for i, (vector, origin, color) in enumerate(
zip(vectors, origins, colors, strict=False)
):
rr.log(
f"arrows/{i}", rr.Arrows3D(vectors=vector, origins=origin, colors=color)
)
# Now clear all of them at once.
rr.log("arrows", rr.Clear(recursive=True)) # or `rr.Clear.recursive()`
```
# Capsules3D
3D capsules; cylinders with hemispherical caps.
Capsules are defined by two endpoints (the centers of their end cap spheres), which are located
at (0, 0, 0) and (0, 0, length), that is, extending along the positive direction of the Z axis.
Capsules in other orientations may be produced by applying a rotation to the entity or
instances.
If there's more instance poses than lengths & radii, the last capsule's orientation will be repeated for the remaining poses.
Orienting and placing capsules forms a separate transform that is applied prior to [`archetypes.InstancePoses3D`](https://rerun.io/docs/reference/types/archetypes/instance_poses3d) and [`archetypes.Transform3D`](https://rerun.io/docs/reference/types/archetypes/transform3d).
## Fields
### Required
* `lengths`: [`Length`](https://rerun.io/docs/reference/types/components/length.md)
* `radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
### Recommended
* `translations`: [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* `colors`: [`Color`](https://rerun.io/docs/reference/types/components/color.md)
### Optional
* `rotation_axis_angles`: [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
* `quaternions`: [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
* `line_radii`: [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* `fill_mode`: [`FillMode`](https://rerun.io/docs/reference/types/components/fill_mode.md)
* `labels`: [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* `show_labels`: [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* `class_ids`: [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
## Can be shown in
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md)
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md) (if logged above active projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Capsules3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Capsules3D.html)
* π [Python API docs for `Capsules3D`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Capsules3D)
* π¦ [Rust API docs for `Capsules3D`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Capsules3D.html)
## Example
### Batch of capsules
```python
"""Log a batch of capsules."""
import rerun as rr
rr.init("rerun_example_capsule3d_batch", spawn=True)
rr.log(
"capsules",
rr.Capsules3D(
lengths=[0.0, 2.0, 4.0, 6.0, 8.0],
radii=[1.0, 0.5, 0.5, 0.5, 1.0],
colors=[
(255, 0, 0),
(188, 188, 0),
(0, 255, 0),
(0, 188, 188),
(0, 0, 255),
],
translations=[
(0.0, 0.0, 0.0),
(2.0, 0.0, 0.0),
(4.0, 0.0, 0.0),
(6.0, 0.0, 0.0),
(8.0, 0.0, 0.0),
],
rotation_axis_angles=[
rr.RotationAxisAngle(
[1.0, 0.0, 0.0],
rr.Angle(deg=float(i) * -22.5),
)
for i in range(5)
],
),
)
```
# Scalars
One or more double-precision scalar values, e.g. for use for time-series plots.
The current timeline value will be used for the time/X-axis, hence scalars
should not be static.
Number of scalars per timestamp is expected to be the same over time.
When used to produce a plot, this archetype is used to provide the data that
is referenced by [`archetypes.SeriesLines`](https://rerun.io/docs/reference/types/archetypes/series_lines) or [`archetypes.SeriesPoints`](https://rerun.io/docs/reference/types/archetypes/series_points). You can do
this by logging both archetypes to the same path, or alternatively configuring
the plot-specific archetypes through the blueprint.
## Fields
### Required
* `scalars`: [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md)
## Can be shown in
* [TimeSeriesView](https://rerun.io/docs/reference/types/views/time_series_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `Scalars`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1Scalars.html)
* π [Python API docs for `Scalars`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.Scalars)
* π¦ [Rust API docs for `Scalars`](https://docs.rs/rerun/latest/rerun/archetypes/struct.Scalars.html)
## Examples
### Simple line plot
```python
"""Log a scalar over time."""
import math
import rerun as rr
rr.init("rerun_example_scalar", spawn=True)
# Log the data on a timeline called "step".
for step in range(64):
rr.set_time("step", sequence=step)
rr.log("scalar", rr.Scalars(math.sin(step / 10.0)))
```
### Multiple time series plots
```python
"""Log a scalar over time."""
from math import cos, sin, tau
import numpy as np
import rerun as rr
rr.init("rerun_example_scalar_multiple_plots", spawn=True)
lcg_state = np.int64(0)
# Set up plot styling:
# They are logged as static as they don't change over time and apply to
# all timelines.
# Log two lines series under a shared root so that they show in the same
# plot by default.
rr.log(
"trig/sin",
rr.SeriesLines(colors=[255, 0, 0], names="sin(0.01t)"),
static=True,
)
rr.log(
"trig/cos",
rr.SeriesLines(colors=[0, 255, 0], names="cos(0.01t)"),
static=True,
)
# NOTE: `SeriesLines` and `SeriesPoints` can both be logged without any
# associated data (all fields are optional). In `v0.24` we removed
# indicators, which now results in no data logged at all, when no
# fields are specified. Therefore, we log a circle shape as a
# marker if no arguments are supplied.
# More information: https://github.com/rerun-io/rerun/issues/10512
# Log scattered points under a different root so that they show in a
# different plot by default.
rr.log("scatter/lcg", rr.SeriesPoints(), static=True)
# Log the data on a timeline called "step".
for t in range(int(tau * 2 * 100.0)):
rr.set_time("step", sequence=t)
rr.log("trig/sin", rr.Scalars(sin(float(t) / 100.0)))
rr.log("trig/cos", rr.Scalars(cos(float(t) / 100.0)))
# simple linear congruency generator
lcg_state = (1140671485 * lcg_state + 128201163) % 16777216
rr.log("scatter/lcg", rr.Scalars(lcg_state.astype(np.float64)))
```
### Update a scalar over time
```python
"""
Update a scalar over time.
See also the `scalar_column_updates` example, which achieves the same
thing in a single operation.
"""
from __future__ import annotations
import math
import rerun as rr
rr.init("rerun_example_scalar_row_updates", spawn=True)
for step in range(64):
rr.set_time("step", sequence=step)
rr.log("scalars", rr.Scalars(math.sin(step / 10.0)))
```
### Update a scalar over time, in a single operation
```python
"""
Update a scalar over time, in a single operation.
This is semantically equivalent to the `scalar_row_updates` example,
albeit much faster.
"""
from __future__ import annotations
import numpy as np
import rerun as rr
rr.init("rerun_example_scalar_column_updates", spawn=True)
times = np.arange(0, 64)
scalars = np.sin(times / 10.0)
rr.send_columns(
"scalars",
indexes=[rr.TimeColumn("step", sequence=times)],
columns=rr.Scalars.columns(scalars=scalars),
)
```
# GraphEdges
A list of edges in a graph.
By default, edges are undirected.
## Fields
### Required
* `edges`: [`GraphEdge`](https://rerun.io/docs/reference/types/components/graph_edge.md)
### Recommended
* `graph_type`: [`GraphType`](https://rerun.io/docs/reference/types/components/graph_type.md)
## Can be shown in
* [GraphView](https://rerun.io/docs/reference/types/views/graph_view.md)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `GraphEdges`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1GraphEdges.html)
* π [Python API docs for `GraphEdges`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.GraphEdges)
* π¦ [Rust API docs for `GraphEdges`](https://docs.rs/rerun/latest/rerun/archetypes/struct.GraphEdges.html)
## Examples
### Simple undirected graph
```python
"""Log a simple undirected graph."""
import rerun as rr
rr.init("rerun_example_graph_undirected", spawn=True)
rr.log(
"simple",
rr.GraphNodes(
node_ids=["a", "b", "c"],
positions=[(0.0, 100.0), (-100.0, 0.0), (100.0, 0.0)],
labels=["A", "B", "C"],
),
rr.GraphEdges(
edges=[("a", "b"), ("b", "c"), ("c", "a")],
# Optional: graphs are undirected by default.
graph_type="undirected",
),
)
```
### Simple directed graph
```python
"""Log a simple directed graph."""
import rerun as rr
rr.init("rerun_example_graph_directed", spawn=True)
rr.log(
"simple",
rr.GraphNodes(
node_ids=["a", "b", "c"],
positions=[(0.0, 100.0), (-100.0, 0.0), (100.0, 0.0)],
labels=["A", "B", "C"],
),
rr.GraphEdges(
edges=[("a", "b"), ("b", "c"), ("c", "a")], graph_type="directed"
),
)
```
# SegmentationImage
An image made up of integer [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id)s.
Each pixel corresponds to a [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id) that will be mapped to a color based on [`archetypes.AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context).
In the case of floating point images, the label will be looked up based on rounding to the nearest
integer value.
Use [`archetypes.AnnotationContext`](https://rerun.io/docs/reference/types/archetypes/annotation_context) to associate each class with a color and a label.
## Fields
### Required
* `buffer`: [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* `format`: [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
### Optional
* `opacity`: [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* `draw_order`: [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
## Can be shown in
* [Spatial2DView](https://rerun.io/docs/reference/types/views/spatial2d_view.md)
* [Spatial3DView](https://rerun.io/docs/reference/types/views/spatial3d_view.md) (if logged under a projection)
* [DataframeView](https://rerun.io/docs/reference/types/views/dataframe_view.md)
## API reference links
* π [C++ API docs for `SegmentationImage`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1archetypes_1_1SegmentationImage.html)
* π [Python API docs for `SegmentationImage`](https://ref.rerun.io/docs/python/stable/common/archetypes#rerun.archetypes.SegmentationImage)
* π¦ [Rust API docs for `SegmentationImage`](https://docs.rs/rerun/latest/rerun/archetypes/struct.SegmentationImage.html)
## Example
### Simple segmentation image
```python
"""Create and log a segmentation image."""
import numpy as np
import rerun as rr
# Create a segmentation image
image = np.zeros((8, 12), dtype=np.uint8)
image[0:4, 0:6] = 1
image[4:8, 6:12] = 2
rr.init("rerun_example_segmentation_image", spawn=True)
# Assign a label and color to each class
rr.log(
"/",
rr.AnnotationContext([(1, "red", (255, 0, 0)), (2, "green", (0, 255, 0))]),
static=True,
)
rr.log("image", rr.SegmentationImage(image))
```
# Mat3x3
A 3x3 Matrix.
Matrices in Rerun are stored as flat list of coefficients in column-major order:
```text
column 0 column 1 column 2
-------------------------------------------------
row 0 | flat_columns[0] flat_columns[3] flat_columns[6]
row 1 | flat_columns[1] flat_columns[4] flat_columns[7]
row 2 | flat_columns[2] flat_columns[5] flat_columns[8]
```
## Arrow datatype
```
FixedSizeList(9 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Mat3x3`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Mat3x3.html)
* π [Python API docs for `Mat3x3`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Mat3x3)
* π¦ [Rust API docs for `Mat3x3`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Mat3x3.html)
## Used by
* [`PinholeProjection`](https://rerun.io/docs/reference/types/components/pinhole_projection.md)
* [`TransformMat3x3`](https://rerun.io/docs/reference/types/components/transform_mat3x3.md)
# TensorDimensionSelection
Selection of a single tensor dimension.
## Fields
#### `dimension`
Type: non-null `UInt32`
The dimension number to select.
#### `invert`
Type: non-null `Boolean`
Invert the direction of the dimension.
## Arrow datatype
```
Struct(
"dimension": non-null UInt32
"invert": non-null Boolean
)
```
## API reference links
* π [C++ API docs for `TensorDimensionSelection`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TensorDimensionSelection.html)
* π [Python API docs for `TensorDimensionSelection`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TensorDimensionSelection)
* π¦ [Rust API docs for `TensorDimensionSelection`](https://docs.rs/rerun/latest/rerun/datatypes/struct.TensorDimensionSelection.html)
## Used by
* [`TensorHeightDimension`](https://rerun.io/docs/reference/types/components/tensor_height_dimension.md)
* [`TensorWidthDimension`](https://rerun.io/docs/reference/types/components/tensor_width_dimension.md)
# KeypointId
A 16-bit ID representing a type of semantic keypoint within a class.
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `KeypointId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1KeypointId.html)
* π [Python API docs for `KeypointId`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.KeypointId)
* π¦ [Rust API docs for `KeypointId`](https://docs.rs/rerun/latest/rerun/datatypes/struct.KeypointId.html)
## Used by
* [`KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id.md)
* [`KeypointPair`](https://rerun.io/docs/reference/types/datatypes/keypoint_pair.md)
# TimeRange
Visible time range bounds for a specific timeline.
## Fields
#### `start`
Type: non-null [`TimeRangeBoundary`](https://rerun.io/docs/reference/types/datatypes/time_range_boundary.md)
Low time boundary for sequence timeline.
#### `end`
Type: non-null [`TimeRangeBoundary`](https://rerun.io/docs/reference/types/datatypes/time_range_boundary.md)
High time boundary for sequence timeline.
## Arrow datatype
```
Struct(
"start": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("CursorRelative": non-null Int64)
2: ("Absolute": non-null Int64)
3: ("Infinite": Null)
)
"end": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("CursorRelative": non-null Int64)
2: ("Absolute": non-null Int64)
3: ("Infinite": Null)
)
)
```
## API reference links
* π [C++ API docs for `TimeRange`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TimeRange.html)
* π [Python API docs for `TimeRange`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TimeRange)
* π¦ [Rust API docs for `TimeRange`](https://docs.rs/rerun/latest/rerun/datatypes/struct.TimeRange.html)
## Used by
* [`VisibleTimeRange`](https://rerun.io/docs/reference/types/datatypes/visible_time_range.md)
# Range1D
A 1D range, specifying a lower and upper bound.
## Arrow datatype
```
FixedSizeList(2 x non-null Float64)
```
## API reference links
* π [C++ API docs for `Range1D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Range1D.html)
* π [Python API docs for `Range1D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Range1D)
* π¦ [Rust API docs for `Range1D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Range1D.html)
## Used by
* [`Range1D`](https://rerun.io/docs/reference/types/components/range1d.md)
* [`Range2D`](https://rerun.io/docs/reference/types/datatypes/range2d.md)
* [`ValueRange`](https://rerun.io/docs/reference/types/components/value_range.md)
# UVec3D
A uint32 vector in 3D space.
## Arrow datatype
```
FixedSizeList(3 x non-null UInt32)
```
## API reference links
* π [C++ API docs for `UVec3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UVec3D.html)
* π [Python API docs for `UVec3D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UVec3D)
* π¦ [Rust API docs for `UVec3D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UVec3D.html)
## Used by
* [`TriangleIndices`](https://rerun.io/docs/reference/types/components/triangle_indices.md)
# Float32
A single-precision 32-bit IEEE 754 floating point number.
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `Float32`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Float32.html)
* π [Python API docs for `Float32`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Float32)
* π¦ [Rust API docs for `Float32`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Float32.html)
## Used by
* [`AxisLength`](https://rerun.io/docs/reference/types/components/axis_length.md)
* [`CellSize`](https://rerun.io/docs/reference/types/components/cell_size.md)
* [`DepthMeter`](https://rerun.io/docs/reference/types/components/depth_meter.md)
* [`DrawOrder`](https://rerun.io/docs/reference/types/components/draw_order.md)
* [`FillRatio`](https://rerun.io/docs/reference/types/components/fill_ratio.md)
* [`GammaCorrection`](https://rerun.io/docs/reference/types/components/gamma_correction.md)
* [`ImagePlaneDistance`](https://rerun.io/docs/reference/types/components/image_plane_distance.md)
* [`Length`](https://rerun.io/docs/reference/types/components/length.md)
* [`MarkerSize`](https://rerun.io/docs/reference/types/components/marker_size.md)
* [`Opacity`](https://rerun.io/docs/reference/types/components/opacity.md)
* [`Radius`](https://rerun.io/docs/reference/types/components/radius.md)
* [`StrokeWidth`](https://rerun.io/docs/reference/types/components/stroke_width.md)
* [`VoxelValue`](https://rerun.io/docs/reference/types/components/voxel_value.md)
# UInt16
A 16bit unsigned integer.
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `UInt16`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UInt16.html)
* π [Python API docs for `UInt16`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UInt16)
* π¦ [Rust API docs for `UInt16`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UInt16.html)
## Used by
* [`ChannelCountPair`](https://rerun.io/docs/reference/types/datatypes/channel_count_pair.md)
* [`ChannelId`](https://rerun.io/docs/reference/types/components/channel_id.md)
* [`SchemaId`](https://rerun.io/docs/reference/types/components/schema_id.md)
# PixelFormat
Specifieds a particular format of an [`archetypes.Image`](https://rerun.io/docs/reference/types/archetypes/image).
Most images can be described by a [`datatypes.ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model) and a [`datatypes.ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype),
e.g. `RGB` and `U8` respectively.
However, some image formats has chroma downsampling and/or
use differing number of bits per channel, and that is what this [`datatypes.PixelFormat`](https://rerun.io/docs/reference/types/datatypes/pixel_format) is for.
All these formats support random access.
For more compressed image formats, see [`archetypes.EncodedImage`](https://rerun.io/docs/reference/types/archetypes/encoded_image).
## Variants
#### `Y_U_V12_LimitedRange` = 20
`Y_U_V12` is a YUV 4:2:0 fully planar YUV format without chroma downsampling, also known as `I420`.
This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
the resolution of the Y plane.
#### `NV12` = 26
`NV12` (aka `Y_UV12`) is a YUV 4:2:0 chroma downsampled form at with 12 bits per pixel and 8 bits per channel.
This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
First comes entire image in Y in one plane,
followed by a plane with interleaved lines ordered as U0, V0, U1, V1, etc.
#### `YUY2` = 27
`YUY2` (aka 'YUYV', 'YUYV16' or 'NV21'), is a YUV 4:2:2 chroma downsampled format with 16 bits per pixel and 8 bits per channel.
This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
The order of the channels is Y0, U0, Y1, V0, all in the same plane.
#### `Y8_FullRange` = 30
Monochrome Y plane only, essentially a YUV 4:0:0 planar format.
Also known as just "gray". This is virtually identical to a 8bit luminance/grayscale (see [`datatypes.ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model)).
This uses entire range YUV, i.e. Y is expected to be within [0, 255].
(as opposed to "limited range" YUV as used e.g. in NV12).
#### `Y_U_V24_LimitedRange` = 39
`Y_U_V24` is a YUV 4:4:4 fully planar YUV format without chroma downsampling, also known as `I444`.
This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
First comes entire image in Y in one plane, followed by the U and V planes.
#### `Y_U_V24_FullRange` = 40
`Y_U_V24` is a YUV 4:4:4 fully planar YUV format without chroma downsampling, also known as `I444`.
This uses full range YUV with all components ranging from 0 to 255
(as opposed to "limited range" YUV as used e.g. in NV12).
First comes entire image in Y in one plane, followed by the U and V planes.
#### `Y8_LimitedRange` = 41
Monochrome Y plane only, essentially a YUV 4:0:0 planar format.
Also known as just "gray".
This uses limited range YUV, i.e. Y is expected to be within [16, 235].
If not for this range limitation/remapping, this is almost identical to 8bit luminace/grayscale (see [`datatypes.ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model)).
#### `Y_U_V12_FullRange` = 44
`Y_U_V12` is a YUV 4:2:0 fully planar YUV format without chroma downsampling, also known as `I420`.
This uses full range YUV with all components ranging from 0 to 255
(as opposed to "limited range" YUV as used e.g. in NV12).
First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
the resolution of the Y plane.
#### `Y_U_V16_LimitedRange` = 49
`Y_U_V16` is a YUV 4:2:2 fully planar YUV format without chroma downsampling, also known as `I422`.
This uses limited range YUV, i.e. Y is expected to be within [16, 235] and U/V within [16, 240].
First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
the horizontal resolution of the Y plane.
#### `Y_U_V16_FullRange` = 50
`Y_U_V16` is a YUV 4:2:2 fully planar YUV format without chroma downsampling, also known as `I422`.
This uses full range YUV with all components ranging from 0 to 255
(as opposed to "limited range" YUV as used e.g. in NV12).
First comes entire image in Y in one plane, followed by the U and V planes, which each only have half
the horizontal resolution of the Y plane.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `PixelFormat`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1datatypes.html)
* π [Python API docs for `PixelFormat`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.PixelFormat)
* π¦ [Rust API docs for `PixelFormat`](https://docs.rs/rerun/latest/rerun/datatypes/enum.PixelFormat.html)
## Used by
* [`ImageFormat`](https://rerun.io/docs/reference/types/datatypes/image_format.md)
# IVec3D
An int32 vector in 3D space.
## Arrow datatype
```
FixedSizeList(3 x non-null Int32)
```
## API reference links
* π [C++ API docs for `IVec3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1IVec3D.html)
* π [Python API docs for `IVec3D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.IVec3D)
* π¦ [Rust API docs for `IVec3D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.IVec3D.html)
## Used by
* [`VoxelIndex`](https://rerun.io/docs/reference/types/components/voxel_index.md)
# VideoTimestamp
Presentation timestamp within a [`archetypes.AssetVideo`](https://rerun.io/docs/reference/types/archetypes/asset_video).
Specified in nanoseconds.
Presentation timestamps are typically measured as time since video start.
## Arrow datatype
```
Int64
```
## API reference links
* π [C++ API docs for `VideoTimestamp`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1VideoTimestamp.html)
* π [Python API docs for `VideoTimestamp`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.VideoTimestamp)
* π¦ [Rust API docs for `VideoTimestamp`](https://docs.rs/rerun/latest/rerun/datatypes/struct.VideoTimestamp.html)
## Used by
* [`VideoTimestamp`](https://rerun.io/docs/reference/types/components/video_timestamp.md)
# ClassDescriptionMapElem
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
A helper type for mapping [`datatypes.ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id)s to class descriptions.
This is internal to [`components.AnnotationContext`](https://rerun.io/docs/reference/types/components/annotation_context).
## Fields
#### `class_id`
Type: non-null [`ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id.md)
The key: the [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id).
#### `class_description`
Type: non-null [`ClassDescription`](https://rerun.io/docs/reference/types/datatypes/class_description.md)
The value: class name, color, etc.
## Arrow datatype
```
Struct(
"class_id": non-null UInt16
"class_description": non-null Struct(
"info": non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
)
"keypoint_annotations": non-null List(non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
))
"keypoint_connections": non-null List(non-null Struct(
"keypoint0": non-null UInt16
"keypoint1": non-null UInt16
))
)
)
```
## API reference links
* π [C++ API docs for `ClassDescriptionMapElem`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ClassDescriptionMapElem.html)
* π [Python API docs for `ClassDescriptionMapElem`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ClassDescriptionMapElem)
* π¦ [Rust API docs for `ClassDescriptionMapElem`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ClassDescriptionMapElem.html)
## Used by
* [`AnnotationContext`](https://rerun.io/docs/reference/types/components/annotation_context.md)
# TensorBuffer
The underlying storage for [`archetypes.Tensor`](https://rerun.io/docs/reference/types/archetypes/tensor).
Tensor elements are stored in a contiguous buffer of a single type.
## Variants
#### `U8` = 1
Type: non-null List of `UInt8`
8bit unsigned integer.
#### `U16` = 2
Type: non-null List of `UInt16`
16bit unsigned integer.
#### `U32` = 3
Type: non-null List of `UInt32`
32bit unsigned integer.
#### `U64` = 4
Type: non-null List of `UInt64`
64bit unsigned integer.
#### `I8` = 5
Type: non-null List of `Int8`
8bit signed integer.
#### `I16` = 6
Type: non-null List of `Int16`
16bit signed integer.
#### `I32` = 7
Type: non-null List of `Int32`
32bit signed integer.
#### `I64` = 8
Type: non-null List of `Int64`
64bit signed integer.
#### `F16` = 9
Type: non-null List of `Float16`
16bit IEEE-754 floating point, also known as `half`.
#### `F32` = 10
Type: non-null List of `Float32`
32bit IEEE-754 floating point, also known as `float` or `single`.
#### `F64` = 11
Type: non-null List of `Float64`
64bit IEEE-754 floating point, also known as `double`.
## Arrow datatype
```
Union(Dense,
0: ("_null_markers": Null)
1: ("U8": non-null List(non-null UInt8))
2: ("U16": non-null List(non-null UInt16))
3: ("U32": non-null List(non-null UInt32))
4: ("U64": non-null List(non-null UInt64))
5: ("I8": non-null List(non-null Int8))
6: ("I16": non-null List(non-null Int16))
7: ("I32": non-null List(non-null Int32))
8: ("I64": non-null List(non-null Int64))
9: ("F16": non-null List(non-null Float16))
10: ("F32": non-null List(non-null Float32))
11: ("F64": non-null List(non-null Float64))
)
```
## API reference links
* π [C++ API docs for `TensorBuffer`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TensorBuffer.html)
* π [Python API docs for `TensorBuffer`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TensorBuffer)
* π¦ [Rust API docs for `TensorBuffer`](https://docs.rs/rerun/latest/rerun/datatypes/enum.TensorBuffer.html)
## Used by
* [`TensorData`](https://rerun.io/docs/reference/types/datatypes/tensor_data.md)
# ImageFormat
The metadata describing the contents of a [`components.ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer).
## Fields
#### `width`
Type: non-null `UInt32`
The width of the image in pixels.
#### `height`
Type: non-null `UInt32`
The height of the image in pixels.
#### `pixel_format`
Type: [`PixelFormat`](https://rerun.io/docs/reference/types/datatypes/pixel_format.md)
Used mainly for chroma downsampled formats and differing number of bits per channel.
If specified, this takes precedence over both [`datatypes.ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model) and [`datatypes.ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype) (which are ignored).
#### `color_model`
Type: [`ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model.md)
L, RGB, RGBA, β¦
Also requires a [`datatypes.ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype) to fully specify the pixel format.
#### `channel_datatype`
Type: [`ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype.md)
The data type of each channel (e.g. the red channel) of the image data (U8, F16, β¦).
Also requires a [`datatypes.ColorModel`](https://rerun.io/docs/reference/types/datatypes/color_model) to fully specify the pixel format.
## Arrow datatype
```
Struct(
"width": non-null UInt32
"height": non-null UInt32
"pixel_format": UInt8
"color_model": UInt8
"channel_datatype": UInt8
)
```
## API reference links
* π [C++ API docs for `ImageFormat`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ImageFormat.html)
* π [Python API docs for `ImageFormat`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ImageFormat)
* π¦ [Rust API docs for `ImageFormat`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ImageFormat.html)
## Used by
* [`ImageFormat`](https://rerun.io/docs/reference/types/components/image_format.md)
# ClassDescription
The description of a semantic Class.
If an entity is annotated with a corresponding [`components.ClassId`](https://rerun.io/docs/reference/types/components/class_id), Rerun will use
the attached [`datatypes.AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info) to derive labels and colors.
Keypoints within an annotation class can similarly be annotated with a
[`components.KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id) in which case we should defer to the label and color for the
[`datatypes.AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info) specifically associated with the Keypoint.
Keypoints within the class can also be decorated with skeletal edges.
Keypoint-connections are pairs of [`components.KeypointId`](https://rerun.io/docs/reference/types/components/keypoint_id)s. If an edge is
defined, and both keypoints exist within the instance of the class, then the
keypoints should be connected with an edge. The edge should be labeled and
colored as described by the class's [`datatypes.AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info).
## Fields
#### `info`
Type: non-null [`AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info.md)
The [`datatypes.AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info) for the class.
#### `keypoint_annotations`
Type: non-null List of [`AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info.md)
The [`datatypes.AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info) for all of the keypoints.
#### `keypoint_connections`
Type: non-null List of [`KeypointPair`](https://rerun.io/docs/reference/types/datatypes/keypoint_pair.md)
The connections between keypoints.
## Arrow datatype
```
Struct(
"info": non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
)
"keypoint_annotations": non-null List(non-null Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
))
"keypoint_connections": non-null List(non-null Struct(
"keypoint0": non-null UInt16
"keypoint1": non-null UInt16
))
)
```
## API reference links
* π [C++ API docs for `ClassDescription`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ClassDescription.html)
* π [Python API docs for `ClassDescription`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ClassDescription)
* π¦ [Rust API docs for `ClassDescription`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ClassDescription.html)
## Used by
* [`ClassDescriptionMapElem`](https://rerun.io/docs/reference/types/datatypes/class_description_map_elem.md)
# Float64
A double-precision 64-bit IEEE 754 floating point number.
## Arrow datatype
```
Float64
```
## API reference links
* π [C++ API docs for `Float64`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Float64.html)
* π [Python API docs for `Float64`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Float64)
* π¦ [Rust API docs for `Float64`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Float64.html)
## Used by
* [`LinearSpeed`](https://rerun.io/docs/reference/types/components/linear_speed.md)
* [`Scalar`](https://rerun.io/docs/reference/types/components/scalar.md)
# Vec2D
A vector in 2D space.
## Arrow datatype
```
FixedSizeList(2 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Vec2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Vec2D.html)
* π [Python API docs for `Vec2D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Vec2D)
* π¦ [Rust API docs for `Vec2D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Vec2D.html)
## Used by
* [`HalfSize2D`](https://rerun.io/docs/reference/types/components/half_size2d.md)
* [`LineStrip2D`](https://rerun.io/docs/reference/types/components/line_strip2d.md)
* [`Position2D`](https://rerun.io/docs/reference/types/components/position2d.md)
* [`Resolution`](https://rerun.io/docs/reference/types/components/resolution.md)
* [`Texcoord2D`](https://rerun.io/docs/reference/types/components/texcoord2d.md)
* [`Vector2D`](https://rerun.io/docs/reference/types/components/vector2d.md)
# Plane3D
An infinite 3D plane represented by a unit normal vector and a distance.
Any point P on the plane fulfills the equation `dot(xyz, P) - d = 0`,
where `xyz` is the plane's normal and `d` the distance of the plane from the origin.
This representation is also known as the Hesse normal form.
Note: although the normal will be passed through to the
datastore as provided, when used in the Viewer, planes will always be normalized.
I.e. the plane with xyz = (2, 0, 0), d = 1 is equivalent to xyz = (1, 0, 0), d = 0.5
## Arrow datatype
```
FixedSizeList(4 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Plane3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Plane3D.html)
* π [Python API docs for `Plane3D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Plane3D)
* π¦ [Rust API docs for `Plane3D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Plane3D.html)
## Used by
* [`Plane3D`](https://rerun.io/docs/reference/types/components/plane3d.md)
# KeypointPair
A connection between two [`datatypes.KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id)s.
## Fields
#### `keypoint0`
Type: non-null [`KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id.md)
The first point of the pair.
#### `keypoint1`
Type: non-null [`KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id.md)
The second point of the pair.
## Arrow datatype
```
Struct(
"keypoint0": non-null UInt16
"keypoint1": non-null UInt16
)
```
## API reference links
* π [C++ API docs for `KeypointPair`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1KeypointPair.html)
* π [Python API docs for `KeypointPair`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.KeypointPair)
* π¦ [Rust API docs for `KeypointPair`](https://docs.rs/rerun/latest/rerun/datatypes/struct.KeypointPair.html)
## Used by
* [`ClassDescription`](https://rerun.io/docs/reference/types/datatypes/class_description.md)
# VisibleTimeRange
Visible time range bounds for a specific timeline.
## Fields
#### `timeline`
Type: non-null [`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
Name of the timeline this applies to.
#### `range`
Type: non-null [`TimeRange`](https://rerun.io/docs/reference/types/datatypes/time_range.md)
Time range to use for this timeline.
## Arrow datatype
```
Struct(
"timeline": non-null Utf8
"range": non-null Struct(
"start": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("CursorRelative": non-null Int64)
2: ("Absolute": non-null Int64)
3: ("Infinite": Null)
)
"end": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("CursorRelative": non-null Int64)
2: ("Absolute": non-null Int64)
3: ("Infinite": Null)
)
)
)
```
## API reference links
* π [C++ API docs for `VisibleTimeRange`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1VisibleTimeRange.html)
* π [Python API docs for `VisibleTimeRange`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.VisibleTimeRange)
* π¦ [Rust API docs for `VisibleTimeRange`](https://docs.rs/rerun/latest/rerun/datatypes/struct.VisibleTimeRange.html)
## Example
### Time-windowed trails (e.g. Trajectories)
```python
"""Log line strips over time and view a sliding window (e.g. trajectories)."""
import math
import rerun as rr
import rerun.blueprint as rrb
def point(t: float, phase: float) -> list[float]:
# Sample a point on a helix.
angle = 0.5 * t + phase
return [math.cos(angle), math.sin(angle), 0.1 * t]
rr.init("rerun_example_line_strips3d_time_window", spawn=True)
# Configure the visible time range in the blueprint.
# You can also override this per entity.
rr.send_blueprint(
rrb.Spatial3DView(
origin="/",
time_ranges=rrb.VisibleTimeRange(
"time",
start=rrb.TimeRangeBoundary.cursor_relative(seconds=-5.0),
end=rrb.TimeRangeBoundary.cursor_relative(),
),
)
)
# Log the line strip increments with timestamps.
for i in range(600):
t0 = i / 30.0
t1 = (i + 1) / 30.0
rr.set_time("time", duration=t1)
rr.log(
"trails",
rr.LineStrips3D(
[
[point(t0, 0.0), point(t1, 0.0)],
[point(t0, math.pi), point(t1, math.pi)],
],
colors=[[255, 120, 0], [0, 180, 255]],
radii=0.02,
),
)
```
# TensorDimensionIndexSelection
Indexing a specific tensor dimension.
Selecting `dimension=2` and `index=42` is similar to doing `tensor[:, :, 42, :, :, β¦]` in numpy.
## Fields
#### `dimension`
Type: non-null `UInt32`
The dimension number to select.
#### `index`
Type: non-null `UInt64`
The index along the dimension to use.
## Arrow datatype
```
Struct(
"dimension": non-null UInt32
"index": non-null UInt64
)
```
## API reference links
* π [C++ API docs for `TensorDimensionIndexSelection`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TensorDimensionIndexSelection.html)
* π [Python API docs for `TensorDimensionIndexSelection`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TensorDimensionIndexSelection)
* π¦ [Rust API docs for `TensorDimensionIndexSelection`](https://docs.rs/rerun/latest/rerun/datatypes/struct.TensorDimensionIndexSelection.html)
## Used by
* [`TensorDimensionIndexSelection`](https://rerun.io/docs/reference/types/components/tensor_dimension_index_selection.md)
# Angle
Angle in radians.
## Arrow datatype
```
Float32
```
## API reference links
* π [C++ API docs for `Angle`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Angle.html)
* π [Python API docs for `Angle`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Angle)
* π¦ [Rust API docs for `Angle`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Angle.html)
## Used by
* [`RotationAxisAngle`](https://rerun.io/docs/reference/types/datatypes/rotation_axis_angle.md)
# Vec3D
A vector in 3D space.
## Arrow datatype
```
FixedSizeList(3 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Vec3D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Vec3D.html)
* π [Python API docs for `Vec3D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Vec3D)
* π¦ [Rust API docs for `Vec3D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Vec3D.html)
## Used by
* [`HalfSize3D`](https://rerun.io/docs/reference/types/components/half_size3d.md)
* [`LineStrip3D`](https://rerun.io/docs/reference/types/components/line_strip3d.md)
* [`Position3D`](https://rerun.io/docs/reference/types/components/position3d.md)
* [`RotationAxisAngle`](https://rerun.io/docs/reference/types/datatypes/rotation_axis_angle.md)
* [`Scale3D`](https://rerun.io/docs/reference/types/components/scale3d.md)
* [`Translation3D`](https://rerun.io/docs/reference/types/components/translation3d.md)
* [`Vector3D`](https://rerun.io/docs/reference/types/components/vector3d.md)
* [`VoxelSize`](https://rerun.io/docs/reference/types/components/voxel_size.md)
# UInt64
A 64bit unsigned integer.
## Arrow datatype
```
UInt64
```
## API reference links
* π [C++ API docs for `UInt64`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UInt64.html)
* π [Python API docs for `UInt64`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UInt64)
* π¦ [Rust API docs for `UInt64`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UInt64.html)
## Used by
* [`ChannelCountPair`](https://rerun.io/docs/reference/types/datatypes/channel_count_pair.md)
* [`Count`](https://rerun.io/docs/reference/types/components/count.md)
# Utf8Pair
Stores a tuple of UTF-8 strings.
## Fields
#### `first`
Type: non-null [`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
The first string.
#### `second`
Type: non-null [`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
The second string.
## Arrow datatype
```
Struct(
"first": non-null Utf8
"second": non-null Utf8
)
```
## API reference links
* π [C++ API docs for `Utf8Pair`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Utf8Pair.html)
* π [Python API docs for `Utf8Pair`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Utf8Pair)
* π¦ [Rust API docs for `Utf8Pair`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Utf8Pair.html)
## Used by
* [`GraphEdge`](https://rerun.io/docs/reference/types/components/graph_edge.md)
* [`KeyValuePairs`](https://rerun.io/docs/reference/types/components/key_value_pairs.md)
# Vec4D
A vector in 4D space.
## Arrow datatype
```
FixedSizeList(4 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Vec4D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Vec4D.html)
* π [Python API docs for `Vec4D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Vec4D)
* π¦ [Rust API docs for `Vec4D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Vec4D.html)
# EntityPath
A path to an entity in the `ChunkStore`.
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `EntityPath`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1EntityPath.html)
* π [Python API docs for `EntityPath`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.EntityPath)
* π¦ [Rust API docs for `EntityPath`](https://docs.rs/rerun/latest/rerun/datatypes/struct.EntityPath.html)
## Used by
* [`EntityPath`](https://rerun.io/docs/reference/types/components/entity_path.md)
# ColorModel
Specified what color components are present in an [`archetypes.Image`](https://rerun.io/docs/reference/types/archetypes/image).
This combined with [`datatypes.ChannelDatatype`](https://rerun.io/docs/reference/types/datatypes/channel_datatype) determines the pixel format of an image.
## Variants
#### `L` = 1
Grayscale luminance intencity/brightness/value, sometimes called `Y`
#### `RGB` = 2
Red, Green, Blue
#### `RGBA` = 3
Red, Green, Blue, Alpha
#### `BGR` = 4
Blue, Green, Red
#### `BGRA` = 5
Blue, Green, Red, Alpha
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `ColorModel`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1datatypes.html)
* π [Python API docs for `ColorModel`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ColorModel)
* π¦ [Rust API docs for `ColorModel`](https://docs.rs/rerun/latest/rerun/datatypes/enum.ColorModel.html)
## Used by
* [`ImageFormat`](https://rerun.io/docs/reference/types/datatypes/image_format.md)
# Bool
A single boolean.
## Arrow datatype
```
Boolean
```
## API reference links
* π [C++ API docs for `Bool`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Bool.html)
* π [Python API docs for `Bool`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Bool)
* π¦ [Rust API docs for `Bool`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Bool.html)
## Used by
* [`ClearIsRecursive`](https://rerun.io/docs/reference/types/components/clear_is_recursive.md)
* [`Interactive`](https://rerun.io/docs/reference/types/components/interactive.md)
* [`IsKeyframe`](https://rerun.io/docs/reference/types/components/is_keyframe.md)
* [`ShowLabels`](https://rerun.io/docs/reference/types/components/show_labels.md)
* [`Visible`](https://rerun.io/docs/reference/types/components/visible.md)
# AbsoluteTimeRange
Two [`datatypes.TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int) describing a range of time.
## Fields
#### `min`
Type: non-null [`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md)
Start of the range.
#### `max`
Type: non-null [`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md)
End of the range (inclusive).
## Arrow datatype
```
Struct(
"min": non-null Int64
"max": non-null Int64
)
```
## API reference links
* π [C++ API docs for `AbsoluteTimeRange`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1AbsoluteTimeRange.html)
* π [Python API docs for `AbsoluteTimeRange`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.AbsoluteTimeRange)
* π¦ [Rust API docs for `AbsoluteTimeRange`](https://docs.rs/rerun/latest/rerun/datatypes/struct.AbsoluteTimeRange.html)
# UInt32
A 32bit unsigned integer.
## Arrow datatype
```
UInt32
```
## API reference links
* π [C++ API docs for `UInt32`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UInt32.html)
* π [Python API docs for `UInt32`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UInt32)
* π¦ [Rust API docs for `UInt32`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UInt32.html)
# Uuid
A 16-byte UUID.
## Arrow datatype
```
FixedSizeList(16 x non-null UInt8)
```
## API reference links
* π [C++ API docs for `Uuid`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Uuid.html)
* π [Python API docs for `Uuid`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Uuid)
* π¦ [Rust API docs for `Uuid`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Uuid.html)
# ViewCoordinates
β οΈ **This type is _unstable_ and may change significantly in a way that the data won't be backwards compatible.**
How we interpret the coordinate system of an entity/space.
For instance: What is "up"? What does the Z axis mean?
The three coordinates are always ordered as [x, y, z].
For example [Right, Down, Forward] means that the X axis points to the right, the Y axis points
down, and the Z axis points forward.
β [Rerun does not yet support left-handed coordinate systems](https://github.com/rerun-io/rerun/issues/5032).
The following constants are used to represent the different directions:
* Up = 1
* Down = 2
* Right = 3
* Left = 4
* Forward = 5
* Back = 6
## Arrow datatype
```
FixedSizeList(3 x non-null UInt8)
```
## API reference links
* π [C++ API docs for `ViewCoordinates`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ViewCoordinates.html)
* π [Python API docs for `ViewCoordinates`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ViewCoordinates)
* π¦ [Rust API docs for `ViewCoordinates`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ViewCoordinates.html)
## Used by
* [`ViewCoordinates`](https://rerun.io/docs/reference/types/components/view_coordinates.md)
# ClassId
A 16-bit ID representing a type of semantic class.
## Arrow datatype
```
UInt16
```
## API reference links
* π [C++ API docs for `ClassId`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ClassId.html)
* π [Python API docs for `ClassId`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ClassId)
* π¦ [Rust API docs for `ClassId`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ClassId.html)
## Used by
* [`ClassDescriptionMapElem`](https://rerun.io/docs/reference/types/datatypes/class_description_map_elem.md)
* [`ClassId`](https://rerun.io/docs/reference/types/components/class_id.md)
# ChannelDatatype
The innermost datatype of an image.
How individual color channel components are encoded.
## Variants
#### `U8` = 6
8-bit unsigned integer.
#### `I8` = 7
8-bit signed integer.
#### `U16` = 8
16-bit unsigned integer.
#### `I16` = 9
16-bit signed integer.
#### `U32` = 10
32-bit unsigned integer.
#### `I32` = 11
32-bit signed integer.
#### `U64` = 12
64-bit unsigned integer.
#### `I64` = 13
64-bit signed integer.
#### `F16` = 33
16-bit IEEE-754 floating point, also known as `half`.
#### `F32` = 34
32-bit IEEE-754 floating point, also known as `float` or `single`.
#### `F64` = 35
64-bit IEEE-754 floating point, also known as `double`.
## Arrow datatype
```
UInt8
```
## API reference links
* π [C++ API docs for `ChannelDatatype`](https://ref.rerun.io/docs/cpp/stable/namespacererun_1_1datatypes.html)
* π [Python API docs for `ChannelDatatype`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ChannelDatatype)
* π¦ [Rust API docs for `ChannelDatatype`](https://docs.rs/rerun/latest/rerun/datatypes/enum.ChannelDatatype.html)
## Used by
* [`ImageFormat`](https://rerun.io/docs/reference/types/datatypes/image_format.md)
# AnnotationInfo
Annotation info annotating a class id or key-point id.
Color and label will be used to annotate entities/keypoints which reference the id.
The id refers either to a class or key-point id
## Fields
#### `id`
Type: non-null `UInt16`
[`datatypes.ClassId`](https://rerun.io/docs/reference/types/datatypes/class_id) or [`datatypes.KeypointId`](https://rerun.io/docs/reference/types/datatypes/keypoint_id) to which this annotation info belongs.
#### `label`
Type: [`Utf8`](https://rerun.io/docs/reference/types/datatypes/utf8.md)
The label that will be shown in the UI.
#### `color`
Type: [`Rgba32`](https://rerun.io/docs/reference/types/datatypes/rgba32.md)
The color that will be applied to the annotated entity.
## Arrow datatype
```
Struct(
"id": non-null UInt16
"label": Utf8
"color": UInt32
)
```
## API reference links
* π [C++ API docs for `AnnotationInfo`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1AnnotationInfo.html)
* π [Python API docs for `AnnotationInfo`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.AnnotationInfo)
* π¦ [Rust API docs for `AnnotationInfo`](https://docs.rs/rerun/latest/rerun/datatypes/struct.AnnotationInfo.html)
## Used by
* [`ClassDescription`](https://rerun.io/docs/reference/types/datatypes/class_description.md)
# UVec2D
A uint32 vector in 2D space.
## Arrow datatype
```
FixedSizeList(2 x non-null UInt32)
```
## API reference links
* π [C++ API docs for `UVec2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UVec2D.html)
* π [Python API docs for `UVec2D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UVec2D)
* π¦ [Rust API docs for `UVec2D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UVec2D.html)
# Quaternion
A Quaternion represented by 4 real numbers.
Note: although the x,y,z,w components of the quaternion will be passed through to the
datastore as provided, when used in the Viewer Quaternions will always be normalized.
## Arrow datatype
```
FixedSizeList(4 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Quaternion`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Quaternion.html)
* π [Python API docs for `Quaternion`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Quaternion)
* π¦ [Rust API docs for `Quaternion`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Quaternion.html)
## Used by
* [`RotationQuat`](https://rerun.io/docs/reference/types/components/rotation_quat.md)
# TimeInt
A 64-bit number describing either nanoseconds OR sequence numbers.
## Arrow datatype
```
Int64
```
## API reference links
* π [C++ API docs for `TimeInt`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TimeInt.html)
* π [Python API docs for `TimeInt`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TimeInt)
* π¦ [Rust API docs for `TimeInt`](https://docs.rs/rerun/latest/rerun/datatypes/struct.TimeInt.html)
## Used by
* [`AbsoluteTimeRange`](https://rerun.io/docs/reference/types/datatypes/absolute_time_range.md)
* [`TimeRangeBoundary`](https://rerun.io/docs/reference/types/datatypes/time_range_boundary.md)
* [`Timestamp`](https://rerun.io/docs/reference/types/components/timestamp.md)
# Rgba32
An RGBA color with unmultiplied/separate alpha, in sRGB gamma space with linear alpha.
The color is stored as a 32-bit integer, where the most significant
byte is `R` and the least significant byte is `A`.
## Arrow datatype
```
UInt32
```
## API reference links
* π [C++ API docs for `Rgba32`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Rgba32.html)
* π [Python API docs for `Rgba32`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Rgba32)
* π¦ [Rust API docs for `Rgba32`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Rgba32.html)
## Used by
* [`AlbedoFactor`](https://rerun.io/docs/reference/types/components/albedo_factor.md)
* [`AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info.md)
* [`Color`](https://rerun.io/docs/reference/types/components/color.md)
# Utf8
A string of text, encoded as UTF-8.
## Arrow datatype
```
Utf8
```
## API reference links
* π [C++ API docs for `Utf8`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Utf8.html)
* π [Python API docs for `Utf8`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Utf8)
* π¦ [Rust API docs for `Utf8`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Utf8.html)
## Used by
* [`AnnotationInfo`](https://rerun.io/docs/reference/types/datatypes/annotation_info.md)
* [`GraphNode`](https://rerun.io/docs/reference/types/components/graph_node.md)
* [`MediaType`](https://rerun.io/docs/reference/types/components/media_type.md)
* [`Name`](https://rerun.io/docs/reference/types/components/name.md)
* [`TextLogLevel`](https://rerun.io/docs/reference/types/components/text_log_level.md)
* [`Text`](https://rerun.io/docs/reference/types/components/text.md)
* [`TransformFrameId`](https://rerun.io/docs/reference/types/components/transform_frame_id.md)
* [`Utf8Pair`](https://rerun.io/docs/reference/types/datatypes/utf8pair.md)
* [`VisibleTimeRange`](https://rerun.io/docs/reference/types/datatypes/visible_time_range.md)
# Blob
A binary blob of data.
## Arrow datatype
```
List(non-null UInt8)
```
## API reference links
* π [C++ API docs for `Blob`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Blob.html)
* π [Python API docs for `Blob`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Blob)
* π¦ [Rust API docs for `Blob`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Blob.html)
## Used by
* [`Blob`](https://rerun.io/docs/reference/types/components/blob.md)
* [`ImageBuffer`](https://rerun.io/docs/reference/types/components/image_buffer.md)
* [`VideoSample`](https://rerun.io/docs/reference/types/components/video_sample.md)
# UVec4D
A uint vector in 4D space.
## Arrow datatype
```
FixedSizeList(4 x non-null UInt32)
```
## API reference links
* π [C++ API docs for `UVec4D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1UVec4D.html)
* π [Python API docs for `UVec4D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.UVec4D)
* π¦ [Rust API docs for `UVec4D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.UVec4D.html)
# Range2D
An Axis-Aligned Bounding Box in 2D space, implemented as the minimum and maximum corners.
## Fields
#### `x_range`
Type: non-null [`Range1D`](https://rerun.io/docs/reference/types/datatypes/range1d.md)
The range of the X-axis (usually left and right bounds).
#### `y_range`
Type: non-null [`Range1D`](https://rerun.io/docs/reference/types/datatypes/range1d.md)
The range of the Y-axis (usually top and bottom bounds).
## Arrow datatype
```
Struct(
"x_range": non-null FixedSizeList(2 x non-null Float64)
"y_range": non-null FixedSizeList(2 x non-null Float64)
)
```
## API reference links
* π [C++ API docs for `Range2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Range2D.html)
* π [Python API docs for `Range2D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Range2D)
* π¦ [Rust API docs for `Range2D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Range2D.html)
# TimeRangeBoundary
Left or right boundary of a time range.
## Variants
#### `CursorRelative` = 1
Type: non-null [`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md)
Boundary is a value relative to the time cursor.
#### `Absolute` = 2
Type: non-null [`TimeInt`](https://rerun.io/docs/reference/types/datatypes/time_int.md)
Boundary is an absolute value.
#### `Infinite` = 3
Type: `null`
The boundary extends to infinity.
## Arrow datatype
```
Union(Dense,
0: ("_null_markers": Null)
1: ("CursorRelative": non-null Int64)
2: ("Absolute": non-null Int64)
3: ("Infinite": Null)
)
```
## API reference links
* π [C++ API docs for `TimeRangeBoundary`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TimeRangeBoundary.html)
* π [Python API docs for `TimeRangeBoundary`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TimeRangeBoundary)
* π¦ [Rust API docs for `TimeRangeBoundary`](https://docs.rs/rerun/latest/rerun/datatypes/enum.TimeRangeBoundary.html)
## Used by
* [`TimeRange`](https://rerun.io/docs/reference/types/datatypes/time_range.md)
# DVec2D
A double-precision vector in 2D space.
## Arrow datatype
```
FixedSizeList(2 x non-null Float64)
```
## API reference links
* π [C++ API docs for `DVec2D`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1DVec2D.html)
* π [Python API docs for `DVec2D`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.DVec2D)
* π¦ [Rust API docs for `DVec2D`](https://docs.rs/rerun/latest/rerun/datatypes/struct.DVec2D.html)
## Used by
* [`GeoLineString`](https://rerun.io/docs/reference/types/components/geo_line_string.md)
* [`LatLon`](https://rerun.io/docs/reference/types/components/lat_lon.md)
# ChannelCountPair
A pair representing a channel ID and its associated message count.
## Fields
#### `channel_id`
Type: non-null [`UInt16`](https://rerun.io/docs/reference/types/datatypes/uint16.md)
The channel ID.
#### `message_count`
Type: non-null [`UInt64`](https://rerun.io/docs/reference/types/datatypes/uint64.md)
The message count for this channel.
## Arrow datatype
```
Struct(
"channel_id": non-null UInt16
"message_count": non-null UInt64
)
```
## API reference links
* π [C++ API docs for `ChannelCountPair`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1ChannelCountPair.html)
* π [Python API docs for `ChannelCountPair`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.ChannelCountPair)
* π¦ [Rust API docs for `ChannelCountPair`](https://docs.rs/rerun/latest/rerun/datatypes/struct.ChannelCountPair.html)
## Used by
* [`ChannelMessageCounts`](https://rerun.io/docs/reference/types/components/channel_message_counts.md)
# RotationAxisAngle
3D rotation represented by a rotation around a given axis.
## Fields
#### `axis`
Type: non-null [`Vec3D`](https://rerun.io/docs/reference/types/datatypes/vec3d.md)
Axis to rotate around.
This is not required to be normalized.
However, if normalization of the rotation axis fails (typically due to a zero vector)
the rotation is treated as an invalid transform, unless the angle is zero in which case
it is treated as an identity.
#### `angle`
Type: non-null [`Angle`](https://rerun.io/docs/reference/types/datatypes/angle.md)
How much to rotate around the axis.
## Arrow datatype
```
Struct(
"axis": non-null FixedSizeList(3 x non-null Float32)
"angle": non-null Float32
)
```
## API reference links
* π [C++ API docs for `RotationAxisAngle`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1RotationAxisAngle.html)
* π [Python API docs for `RotationAxisAngle`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.RotationAxisAngle)
* π¦ [Rust API docs for `RotationAxisAngle`](https://docs.rs/rerun/latest/rerun/datatypes/struct.RotationAxisAngle.html)
## Used by
* [`RotationAxisAngle`](https://rerun.io/docs/reference/types/components/rotation_axis_angle.md)
# TensorData
An N-dimensional array of numbers.
The number of dimensions and their respective lengths is specified by the `shape` field.
The dimensions are ordered from outermost to innermost. For example, in the common case of
a 2D RGB Image, the shape would be `[height, width, channel]`.
These dimensions are combined with an index to look up values from the `buffer` field,
which stores a contiguous array of typed values.
## Fields
#### `shape`
Type: non-null List of `UInt64`
The shape of the tensor, i.e. the length of each dimension.
#### `names`
Type: List of `Utf8`
The names of the dimensions of the tensor (optional).
If set, should be the same length as [`datatypes.TensorData#shape`](https://rerun.io/docs/reference/types/datatypes/tensor_data).
If it has a different length your names may show up improperly,
and some constructors may produce a warning or even an error.
Example: `["height", "width", "channel", "batch"]`.
#### `buffer`
Type: non-null [`TensorBuffer`](https://rerun.io/docs/reference/types/datatypes/tensor_buffer.md)
The content/data.
## Arrow datatype
```
Struct(
"shape": non-null List(non-null UInt64)
"names": List(non-null Utf8)
"buffer": non-null Union(Dense,
0: ("_null_markers": Null)
1: ("U8": non-null List(non-null UInt8))
2: ("U16": non-null List(non-null UInt16))
3: ("U32": non-null List(non-null UInt32))
4: ("U64": non-null List(non-null UInt64))
5: ("I8": non-null List(non-null Int8))
6: ("I16": non-null List(non-null Int16))
7: ("I32": non-null List(non-null Int32))
8: ("I64": non-null List(non-null Int64))
9: ("F16": non-null List(non-null Float16))
10: ("F32": non-null List(non-null Float32))
11: ("F64": non-null List(non-null Float64))
)
)
```
## API reference links
* π [C++ API docs for `TensorData`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1TensorData.html)
* π [Python API docs for `TensorData`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.TensorData)
* π¦ [Rust API docs for `TensorData`](https://docs.rs/rerun/latest/rerun/datatypes/struct.TensorData.html)
## Used by
* [`TensorData`](https://rerun.io/docs/reference/types/components/tensor_data.md)
# Mat4x4
A 4x4 Matrix.
Matrices in Rerun are stored as flat list of coefficients in column-major order:
```text
column 0 column 1 column 2 column 3
--------------------------------------------------------------------
row 0 | flat_columns[0] flat_columns[4] flat_columns[8] flat_columns[12]
row 1 | flat_columns[1] flat_columns[5] flat_columns[9] flat_columns[13]
row 2 | flat_columns[2] flat_columns[6] flat_columns[10] flat_columns[14]
row 3 | flat_columns[3] flat_columns[7] flat_columns[11] flat_columns[15]
```
## Arrow datatype
```
FixedSizeList(16 x non-null Float32)
```
## API reference links
* π [C++ API docs for `Mat4x4`](https://ref.rerun.io/docs/cpp/stable/structrerun_1_1datatypes_1_1Mat4x4.html)
* π [Python API docs for `Mat4x4`](https://ref.rerun.io/docs/python/stable/common/datatypes#rerun.datatypes.Mat4x4)
* π¦ [Rust API docs for `Mat4x4`](https://docs.rs/rerun/latest/rerun/datatypes/struct.Mat4x4.html)