Teleoperation

Joint state and action beside the cameras, in the format policy training reads — from a bimanual robot arm, or from a person wearing a UMI gripper rig.

Action-conditioned policy training. Every frame carries the state the arms were in and the action taken, which is the pair a policy learns from.

11
Delivery files: 4 min
Playable here

Episode 0 · pick and place into the bin · 1/11

These are web previews, not the data. Each clip is downscaled to play in a browser. The delivery files are full resolution and carry every stream the rig recorded.

How this is captured

Video: 640x480 at 30 fps
On every frame: rgb head · rgb left · rgb right · joint state · joint action
Ships as: mp4 · lerobot

What we run in this category

Workplaces: 29
Distinct tasks: 111
Operator jobs: 33
Graded hard: 26%

Measured across the samples published here, not the full shelf.

Configuration	What ships	Ready in	Per month
Egocentric + gripper (UMI)Head-mounted phone + UMI wrist cam · UMI gripper kit, optional Xsens glovesWhere you want the gripper state recorded with the video — a person wearing the rig, not a robot arm.	head.mp4 + wrist.mp4 + imu.csv (6-DoF, 200 Hz) + gripper_state.json + task_label.json	14-21 business days	1,000 h / month200 h in month one

Delivered as LeRobot v3 · MCAP · HDF5 · RLDS · mocap to FBX / BVH / SMPL

Play the set

Three cameras on one clock, which is what a policy reads. Below them, every episode in the set — one task, eleven attempts, and the variation between them is the data.

This set is the robot side: a bimanual follower arm, two arms, seven joints each plus a gripper. The configuration priced above it — Egocentric + gripper (UMI) — is the human side, a person wearing a wrist rig, and it delivers head.mp4 + wrist.mp4 + imu.csv + gripper_state.json instead. Both are teleoperation; they are not the same recording.

Head

Left

Right

Episode 0 · 640×480 · 30 fps · 16-dimensional state and action per frame

All 11 episodes · head camera

Pick up all the items on the table and put them into the bin on the right.

ep 0

ep 1

ep 2

ep 3

ep 4

ep 5

ep 6

ep 7

ep 8

ep 9

ep 10

Counted off the files: 11 episodes, 14,076 frames, 33 videos. The set's ownmeta/info.json declares 10, 13,465 and 30, and a LeRobot loader reading its splits would drop the last episode without saying so.

One episode, opened up

The video is what a person watches. This is what a policy reads: sixteen numbers for the state the arms were in and sixteen for the action commanded, 30 times a second, for all 1,444 frames of episode 0.

Action — what was commandedState — what the arm did48.1s · every 6th frame

Left arm

state[0:7] · 7 channels · not commanded in this episode

Held at zero for the whole episode. The action column is exactly 0.0 on every frame and the state never leaves sensor noise, so this side of the robot was not driven — the rig is bimanual, this recording is not.

Right arm

state[7:14] · 7 channels · drawn on -0.94 to 2.44

[7]

[8]

[9]

[10]

[11]

[12]

[13]

Left gripper

state[14:15] · 1 channel · not commanded in this episode

Right gripper

state[15:16] · 1 channel · drawn on 0.00 to 0.79

[15]

Column indices are the dataset's own, from meta/modality.json: seven joints an arm, one gripper a side, and action laid out identically to state. Each segment is drawn on its own scale — arm joints are radians and comparable with each other, a gripper is not, and one scale across both would flatten every joint to fit it.

11 of 11 samples/3.7 minutes of source/11 with live data

Sort

/samples/clips/teleop-ep00.mp4

Pick and place into the bin