Teleoperation

Joint state and action beside the cameras, in the format policy training reads — from a bimanual robot arm, or from a person wearing a UMI gripper rig.

Action-conditioned policy training. Every frame carries the state the arms were in and the action taken, which is the pair a policy learns from.

Pick brown box and place it into blue bin · session 1 · 1/11

These are web previews, not the data. Each clip is downscaled to play in a browser. The delivery files are full resolution and carry every stream the rig recorded.

How this is captured

Video: 640x480 at 20 fps
On every frame: rgb head · rgb left · rgb right · joint state · joint action
Ships as: mp4 · lerobot

What we run in this category

Workplaces: 22
Distinct tasks: 99
Operator jobs: 43
Graded hard: 21%

Measured across the samples published here, not the full shelf.

Configuration	What ships	Ready in	Per month
Egocentric + gripper (UMI)Head-mounted phone + UMI wrist cam · UMI gripper kit, optional Xsens glovesWhere you want the gripper state recorded with the video — a person wearing the rig, not a robot arm.	head.mp4 + wrist.mp4 + imu.csv (6-DoF, 200 Hz) + gripper_state.json + task_label.json	14-21 business days	1,000 h / month200 h in month one

Delivered as LeRobot v3 · MCAP · HDF5 · RLDS · mocap to FBX / BVH / SMPL

Play the set

Three cameras on one clock, which is what a policy reads. Below them, one episode from each of the 11 sessions — one task, 134 attempts, and the variation between them is the data.

This set is the robot side: bimanual follower arms, seven joints each, with a five-fingered hand on each arm. The configuration priced above it — Egocentric + gripper (UMI) — is the human side, a person wearing a wrist rig. Both are teleoperation; they are not the same recording.

Head

Left

Right

Session 1 · episode 1 · 640×480 · 20 fps · 40-dimensional state and action per frame

All 11 sessions · head camera

Pick brown box and place it into blue bin.

s1 · ep 1

s2 · ep 3

s3 · ep 2

s4 · ep 6

s5 · ep 13

s6 · ep 11

s7 · ep 5

s8 · ep 0

s9 · ep 6

s10 · ep 12

s11 · ep 4

Counted off the files: 11 sessions, 134 episodes, 20,790 frames, 33 videos — 17 minutes of robot time at 20 fps.

One episode, opened up

The video is what a person watches. This is what a policy reads: 40 numbers for the state the robot was in and 40 for the action commanded, 20 times a second, for all 174 frames of episode 1.

Action — what was commandedState — what the arm did8.7s · 174 frames

Left arm

state[0:7] · 7 channels · drawn on -0.76 to 2.41

[0]

[1]

[2]

[3]

[4]

[5]

[6]

Right arm

state[7:14] · 7 channels · not commanded in this episode

Did not move for the whole episode: state and action both stay inside sensor noise, so this side of the robot was not driven — the rig is bimanual, this recording is not.

Left hand

state[14:20] · 6 channels · drawn on 0.00 to 1.57

[14]

[15]

[16]

[17]

[18]

[19]

Right hand

state[20:26] · 6 channels · not commanded in this episode

Did not move for the whole episode: state and action both stay inside sensor noise, so this side of the robot was not driven — the rig is bimanual, this recording is not.

Left wrist position

state[26:29] · 3 channels · action only, drawn on -0.11 to 0.97

[26]

[27]

[28]

Only the command is recorded here. The state columns hold zero and the identity rotation on every frame, so the dataset carries the wrist pose that was asked for, not the one the arm reached.

Left wrist rotation

state[29:33] · 4 channels · action only, drawn on -0.79 to 1.00

[29]

[30]

[31]

[32]

Only the command is recorded here. The state columns hold zero and the identity rotation on every frame, so the dataset carries the wrist pose that was asked for, not the one the arm reached.

Right wrist position

state[33:36] · 3 channels · not commanded in this episode

Did not move for the whole episode: state and action both stay inside sensor noise, so this side of the robot was not driven — the rig is bimanual, this recording is not.

Right wrist rotation

state[36:40] · 4 channels · not commanded in this episode

Did not move for the whole episode: state and action both stay inside sensor noise, so this side of the robot was not driven — the rig is bimanual, this recording is not.

Column order is the dataset's own, from the feature names in meta/info.json: seven joints an arm, six per hand, then a wrist position and orientation for each side, with action laid out identically to state. Each segment is drawn on its own scale — joint angles are radians, a wrist position is metres and an orientation is a quaternion, and one scale across them would flatten every joint to fit the widest.

11 of 11 samples/17.3 minutes of source/0 with live data

Sort

Preview · episode 13, head camera · 640 x 480