Using layers to append data to segments

In the catalog object model, datasets are a collection of segments, which are a collection of layers identified by a name. Layers are immutable, but data can be added to segments by registering other layers with the same recording id but a different layer name. This how-to page provides examples for two ways data can be added to existing datasets through layers.

Note: layers should not be confused with MCAP layers, which serve a different purpose in the context of MCAP file ingestion.

Adding data to existing segments using layers adding-data-to-existing-segments-using-layers

When registering recordings to a dataset, the recordings are assigned the "base" layer name by default.

Let's register a few recordings from the DROID dataset (included in the Rerun repository for testing) to illustrate this:

import rerun as rr

from pathlib import Path

sample_5_path = Path(__file__).parents[4] / "tests" / "assets" / "rrd" / "sample_5"

server = rr.server.Server(datasets={"sample_dataset": sample_5_path})
client = server.client()
dataset = client.get_dataset(name="sample_dataset")

print(
    dataset.segment_table()
    .select(
        "rerun_segment_id",
        "rerun_layer_names",
    )
    .sort("rerun_segment_id")
)

Output:

┌───────────────────────────────────────┬───────────────────┐
│ rerun_segment_id                      ┆ rerun_layer_names │
│ ---                                   ┆ ---               │
│ type: Utf8                            ┆ type: List[Utf8]  │
╞═══════════════════════════════════════╪═══════════════════╡
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ [base]            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ [base]            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ [base]            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_j807b3f8_2023_06_15_13h_42m_56s ┆ [base]            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s ┆ [base]            │
└───────────────────────────────────────┴───────────────────┘

It is possible to append data to existing segments by creating new .rrd files with matching recording IDs, and registering them to the dataset under new layer names. A common workflow is to query existing segment data, compute derived values (such as metrics or embeddings), and add them as new layers.

As an example, we use the dataset registered previously and compute the tracking error (L2 norm between commanded and actual joint positions) of the robotic arm:

import numpy as np
from datafusion import col

# Query action (commanded) and observation (actual) joint positions
joints = dataset.filter_contents(["/action/joint_positions", "/observation/joint_positions"]).reader(index="real_time")

# Compute tracking error: L2 norm of (commanded - actual) joint positions
segment_ids = pa.table(joints.select("rerun_segment_id").distinct())["rerun_segment_id"].to_numpy()
rrd_paths = []

for seg_id in segment_ids:
    # Filter to this segment and collect as a PyArrow table for efficient extraction to NumPy
    segment_data = pa.table(
        joints.filter(col("rerun_segment_id") == seg_id).select(
            "real_time",
            "/action/joint_positions:Scalars:scalars",
            "/observation/joint_positions:Scalars:scalars",
        )
    )

    timestamps = segment_data["real_time"].to_numpy()

    actions = np.vstack(segment_data["/action/joint_positions:Scalars:scalars"].to_numpy())
    observations = np.vstack(segment_data["/observation/joint_positions:Scalars:scalars"].to_numpy())

    # Compute L2 tracking error per timestep
    tracking_error = np.linalg.norm(actions - observations, axis=1)

    # Create derived RRD with tracking error timeline
    rrd_path = TMP_DIR / f"{seg_id}_tracking_error.rrd"
    rrd_paths.append(rrd_path)

    with rr.RecordingStream(application_id="rerun_example_tracking_error", recording_id=seg_id) as rec:
        rec.save(rrd_path)
        rr.send_columns(
            "/derived/tracking_error",
            indexes=[rr.TimeColumn("real_time", timestamp=timestamps)],
            columns=rr.Scalars.columns(scalars=tracking_error),
        )


# Register derived RRDs as a new layer
dataset.register([p.as_uri() for p in rrd_paths], layer_name="tracking_error").wait()

The key steps are:

Query action (commanded) and observation (actual) joint positions from the dataset
For each segment, compute the L2 norm of the difference as tracking error
Create a new .rrd file with the same recording_id as the original segment
Log the derived data using send_columns() for efficient columnar logging
Register all derived .rrd files to the dataset with a "tracking_error" layer name

The "rerun_layer_names" column of the segment table confirms the new layer was added:

segment_table = (
    dataset.segment_table()
    .select(
        "rerun_segment_id",
        "rerun_layer_names",
    )
    .sort("rerun_segment_id")
)
print(segment_table)

Output:

┌───────────────────────────────────────┬────────────────────────┐
│ rerun_segment_id                      ┆ rerun_layer_names      │
│ ---                                   ┆ ---                    │
│ type: Utf8                            ┆ type: List[Utf8]       │
╞═══════════════════════════════════════╪════════════════════════╡
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ [base, tracking_error] │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ [base, tracking_error] │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ [base, tracking_error] │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_j807b3f8_2023_06_15_13h_42m_56s ┆ [base, tracking_error] │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s ┆ [base, tracking_error] │
└───────────────────────────────────────┴────────────────────────┘

For another example of computing derived data, see the Query video streams page.

In the next section, we will demonstrate that the data has indeed been added by querying the dataset again.

Adding properties to segments using layers adding-properties-to-segments-using-layers

In addition to regular Rerun data, layers can be used to add properties to segments. This is useful for tagging segments with derived metadata based on their content.

In this example, we query the tracking error computed in the previous section, calculate the mean error per segment, and create a tracking_good boolean property based on a threshold:

# Query the tracking error we just added and compute a quality metric
from datafusion import functions as F

tracking = dataset.filter_contents(["/derived/tracking_error"]).reader(index="real_time")
quality_stats = pa.table(
    tracking.aggregate(
        col("rerun_segment_id"),
        [F.avg(col("/derived/tracking_error:Scalars:scalars")[0]).alias("mean_error")],
    )
    .with_column("tracking_good", col("mean_error") < 0.13)
    .select("rerun_segment_id", "tracking_good")
)

# Create RRDs with just the property
rrd_paths = []
for seg_id, tracking_good in zip(quality_stats["rerun_segment_id"], quality_stats["tracking_good"]):
    rrd_path = TMP_DIR / f"{seg_id}_quality.rrd"
    rrd_paths.append(rrd_path)

    with rr.RecordingStream(application_id="rerun_example_quality", recording_id=seg_id) as rec:
        rec.save(rrd_path)
        rec.send_property("quality", rr.AnyValues(tracking_good=tracking_good))

# Register as a separate layer
dataset.register([p.as_uri() for p in rrd_paths], layer_name="quality").wait()

The key steps are:

Query the derived tracking error data we just added
Use DataFusion's aggregate() to compute the mean error per segment
Threshold the mean to create a boolean tracking_good property
Create new .rrd files with send_property() to log the property
Register under a separate "quality" layer

The property now appears in the segment table:

# The segment table now shows both layers and the derived property
segment_table = (
    dataset.segment_table()
    .select(
        "rerun_segment_id",
        "rerun_layer_names",
        "property:quality:tracking_good",
    )
    .sort("rerun_segment_id")
)
print(segment_table)

Output:

┌───────────────────────────────────────┬─────────────────────────────────┬────────────────────────────────────┐
│ rerun_segment_id                      ┆ rerun_layer_names               ┆ property:quality:tracking_good     │
│ ---                                   ┆ ---                             ┆ ---                                │
│ type: Utf8                            ┆ type: List[Utf8]                ┆ type: nullable List[nullable bool] │
│                                       ┆                                 ┆ component: tracking_good           │
│                                       ┆                                 ┆ entity_path: /__properties/quality │
│                                       ┆                                 ┆ kind: data                         │
╞═══════════════════════════════════════╪═════════════════════════════════╪════════════════════════════════════╡
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ [base, tracking_error, quality] ┆ [false]                            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ [base, tracking_error, quality] ┆ [false]                            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ [base, tracking_error, quality] ┆ [true]                             │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_j807b3f8_2023_06_15_13h_42m_56s ┆ [base, tracking_error, quality] ┆ [true]                             │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_sbd7d2c6_2023_12_24_16h_20m_37s ┆ [base, tracking_error, quality] ┆ [true]                             │
└───────────────────────────────────────┴─────────────────────────────────┴────────────────────────────────────┘

See this page for a deep dive into properties and segment tables.

FAQ faq

Can I modify a layer after it has been registered? can-i-modify-a-layer-after-it-has-been-registered

No. Layers are immutable.

What happens if I register to an existing layer name? what-happens-if-i-register-to-an-existing-layer-name

Registering a .rrd file with a recording_id and layer_name that already exists will result in an error.

How can I replace an existing layer? how-can-i-replace-an-existing-layer

There is currently no way to replace an existing layer using the Python SDK. The current workaround consists of recreating the dataset.

Can I query a single layer with the dataframe query? can-i-query-a-single-layer-with-the-dataframe-query

No. Segments are considered an aggregation of all their layers. There is currently no way to query data from a single layer only.

Must layers be registered to all segments in a dataset? must-layers-be-registered-to-all-segments-in-a-dataset

No. Each segment can have its own set of layers. Some segments may have additional layers that others do not.

What is the default layer name? what-is-the-default-layer-name

When you register a recording without specifying a layer_name, it is assigned to the "base" layer.

Is it possible to obtain a dataframe with a list of all layers in a dataset? is-it-possible-to-obtain-a-dataframe-with-a-list-of-all-layers-in-a-dataset

Yes. The DatasetEntry.manifest() method returns a DataFusion DataFrame containing the full dataset manifest, which includes layer information for each segment:

manifest = (
    dataset.manifest()
    .select(
        "rerun_segment_id",
        "rerun_layer_name",
        "property:quality:tracking_good",
    )
    .sort("rerun_segment_id", "rerun_layer_name")
)
print(manifest)

Output:

┌───────────────────────────────────────┬──────────────────┬────────────────────────────────────┐
│ rerun_segment_id                      ┆ rerun_layer_name ┆ property:quality:tracking_good     │
│ ---                                   ┆ ---              ┆ ---                                │
│ type: Utf8                            ┆ type: Utf8       ┆ type: nullable List[nullable bool] │
│                                       ┆                  ┆ component: tracking_good           │
│                                       ┆                  ┆ entity_path: /__properties/quality │
│                                       ┆                  ┆ kind: data                         │
╞═══════════════════════════════════════╪══════════════════╪════════════════════════════════════╡
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ base             ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ quality          ┆ [false]                            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_50aee79f_2023_07_12_20h_55m_08s ┆ tracking_error   ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ base             ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ quality          ┆ [false]                            │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_20_10h_40m_10s ┆ tracking_error   ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ base             ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ quality          ┆ [true]                             │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_5e938e3b_2023_07_28_11h_25m_26s ┆ tracking_error   ┆ null                               │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ ILIAD_j807b3f8_2023_06_15_13h_42m_56s ┆ base             ┆ null                               │
└───────────────────────────────────────┴──────────────────┴────────────────────────────────────┘