Will One AI Model Rule Robotics? Not Until the Data Catches Up

Some of Unitree's founders once took Qualcomm's Vinesh Sukumar to a football field to watch their robots play. Blue robots, red robots. Even when the robots were standing, he noticed, "they keep moving their arms." The robots had learned from human data, he was told, and in human data the arms move along with the legs. So he asked whether they had data "to really make sure that it does not do it." He thinks they're working towards it.

Sukumar told that story on October 9 at the AI Frontier Summit in Santa Clara, during SF Tech Week, on a panel asked whether one AI model will rule robotics. Our CEO, Tam Le, was on the same panel. We think whichever model wins will inherit the habits of its data, the way Unitree's robots did. Tbrain is a robotics data foundry: we capture real work across Vietnam and label it down to the frame, so we have a stake in the answer.

• Robotics has "no ChatGPT moment yet." Panelists split on one model versus many, and those who spoke about data all said it matters.
• General models can now plan robot tasks. In Robocurve's test, GPT-6 Astra still fit a puzzle piece only 2 times in 20.
• Tbrain records real work across Vietnam, from garment lines to salons and engine bays, with 3D hand pose that says which frames are measured and which are guessed, and a QC report on every delivery. Teams building VLAs and world models already work with us.

The panel split on one model versus many

An audience member asked the panel whether one approach will take it all or whether all of them are needed. Wei Xu, chief scientist at Horizon Robotics, wants two parts today: a fast, precise "system one" for manipulation, and an Astra-class model on top for planning and memory that calls on the skills the robot already has. In the long run he expects a single model, the way computer vision, language understanding and image generation each had their own models and then merged, because separate models "cannot work in the best way when they are isolated."

Sukumar, who chased a single model in autonomous systems about 15 years ago, was more cautious. There will always be cases, he said, where a single model "will just not work," and wherever a task has to be repeatable and explainable, in consumer products or anywhere safety matters, "it's going to be tough." Impossible? "I don't think so."

Another panelist argued that the model "doesn't really matter." With enough high-quality data, any model would work, even a simple lookup table. Tam pushed back: the model still matters, for speed and for what it costs to run.

Asked for her own view, Tam borrowed a line she had heard from someone else: there's "no ChatGPT moment yet," no recipe the whole field can see working. Agentic stacks, vision-language-action models (VLAs), vision-language models and others are all racing for it, and "we are at that exploration phase of trying to see what will work, what will stick." She came back to the data: better data, she said, is what will get the field there. Sukumar was blunter: data "really makes a big difference."

The organizers' 11-minute cut of the panel. Start at 7:15 for the lookup-table exchange, 8:27 for the football-field story and 9:14 for Tam's "no ChatGPT moment." Moderated by Xiaoliang Wei, the panel included Tae Myung Huh (UC Santa Cruz), Wei Xu (Horizon Robotics), Vinesh Sukumar (Qualcomm), Ding Zhao (Carnegie Mellon, SafeWorld) and Tam Le (Tbrain). Video: Silicon Valley AI Tea Talk. Thanks to the Silicon Valley AI Pioneer Club and UC Santa Cruz's GenAI Center for hosting, to Xiaoliang Wei for moderating, and to Vinesh Sukumar for the football-field story.

What GPT-6 Astra can and can't do on real robot arms

Earlier in the panel, the moderator brought up OpenAI's GPT-6 Astra, released on September 3, and the videos of people driving robots with it. Tam came at it from the data side. Astra is impressive, she said, but "how does it learn to understand the physical world?" One hypothesis she raised: it has "seen physical world data," human demonstrations among it, so the skill didn't come from language alone. OpenAI's system card says only that Astra "was trained on diverse datasets."

The day after the release, Robocurve put Astra on two YAM robot arms, with the same agent policy it had used for Anthropic's Claude Fable 5.1. Astra dropped a block in a bowl 19 times out of 20; Fable had managed 8, on a different rig. Asked to fit a puzzle piece into its groove, Astra managed 2 of 20, the same as Fable, and stalled at the same final step.

Trials completed out of 20. Fable's bowl runs used a different rig. Data: Robocurve, September 4, 2026.

It's one test, and Robocurve flags its own limits: grading wasn't blinded, and Astra ran two days after Fable. Still, two frontier models stalling at the same last step is worth taking seriously: both planned correctly and failed at the fit, where fingers meet parts and force matters. Speed is the other gap. Kai Williams of Understanding Robots writes that Astra "has to pause for seconds at a time to think about its next movement." That's why Wei Xu wants a fast layer underneath.

Our bet is that this fast layer learns contact from physical data, human and robot, while general models keep doing the planning, at least through 2027. We'll be wrong if, before the end of 2027, a general model fits Robocurve's puzzle piece (Astra's score: 2 of 20) in 15 of 20 tries at real-time speed, while its developer publishes training data with no robot or human-manipulation footage in it.

Robotics has to build its own web of physical data

Language models had the web, written by people for their own reasons. Nobody posts what their hands feel as they thread a needle. Robotics has to collect its own record of touch, force, mistakes and recoveries, and the way that record is collected shapes what robots learn.

The open data robotics leans on today is messier than it looks. Pantheon's open-source checker, Argus, went through 3,546 episodes from nine open datasets and flagged 27% of them for at least one problem of medium severity or worse. Those are automated flags and not yet human-verified, but 27% is a lot. In the datasets with labels, one episode in five had text that didn't fully match the footage. A companion post on popular open gripper and two-arm datasets shows examples, such as a wrist camera that "fully disconnects during the episode" and a camera "covered by a cloth during the episode." A model trained on that learns the glitches along with the task, much as Unitree's robots learned the arm swing that came with their human data. It's why we label how far to trust every frame we deliver.

Human video keeps paying off as it scales. NVIDIA's EgoScale trained on more than 20,000 hours of first-person human video, and its validation loss kept falling as the hours grew, though it needed a small stage of aligned human and robot data at the end. We think detailed labels, like 3D hand pose and named sub-actions, can shrink that stage.

We record real work and label how far to trust each frame

Asked about industry, Wei Xu pointed to work that still isn't automated: "making shoes or making clothes," in environments that are "not well controlled." That's where we work. Our anchor site is a running textile line in Vietnam.

Around it, we capture across Vietnam through networks of industrial and university partners, recruiting, training and QC-ing our own operators, and we record wherever the work already happens: in homes, shops, factories, markets and out in the field. The library covers kitchens and garment lines, engine bays and salons, shop counters, repair benches and stockrooms, recorded by people who do that work for a living. Most of it is medium or hard work, the kind robots still struggle with.

Operators wear our head-mounted capture rigs, which record video, depth and motion data on one hardware clock, through a real shift of their own job, with no robot involved.

One rig, very different jobs: arranging fabric on the textile line, keeping a workshop ledger, untangling wire. Both hands tracked in 3D, 21 joints each, frame by frame.

Hand pose gets the most care, because it's the closest thing to contact that video can carry. Wherever two of the rig's cameras see a hand, we triangulate 21 joints in 3D, in metres, and label that frame measured. Where only one camera can see it, the pose is a single-camera estimate scaled to the wearer's measured hand size, and the frame is labelled guessed. Short gaps are bridged and flagged too.

One hand seen, one hidden. The left hand is triangulated and drawn solid (measured). The right hand is down in the foam where only one camera can see it, so its pose is a single-camera estimate, labelled guessed in the data.

On top of the hand pose, the auto-label pipeline adds body keypoints, object masks, metric depth and the camera's trajectory, and names each sub-action as a verb and a noun. Annotators label failures and how the operator recovered. An AI filter rejects roughly a quarter to a third of raw demonstrations before a person looks at them, and what's left must clear 15 machine-checked hard rules and a three-layer human review.

Those rules exist because of captures like this one. On an ironing session, the auto-labeler's outline of the iron swallowed the towel behind it. The depth map disagreed by 14 cm at the edge, so the capture was flagged, and a reviewer tightened the outline.

Data ships LeRobot- and RLDS-ready, with a Rerun scene for every episode, consent and provenance attached, and a QA report on every delivery, 48 hours or less from raw capture. The same pipeline also covers teleoperated two-arm robot data, studio motion capture and multi-view video.

Teams building VLAs and world models already come to us. For a frontier robotics team pretraining a cross-embodiment VLA, we captured egocentric manipulation in homes, markets and workshops. A world-model lab that had learned from games and simulation needed real, action-paired video to anchor its predictions in physics; we delivered long first-person sequences with synchronized action labels and captions. And when a robotics startup needed cold-start data for a new manipulation task, our pre-QC'd teleoperation and UMI demonstrations were in its training pipeline within days. The case studies have the details.

Where we're pushing next

Robot data is still thin on contact, on mistakes and on the floors labs can't reach, and too much of what exists goes unchecked. That's where we're going next:

• Touch and force. Robocurve's puzzle piece suggests contact is where the gap is, so touch and force are the next layer on top of our video, depth and 3D hand pose. We plan to pilot tactile and force sensing synced with our hand-pose capture alongside university research partners, and to publish what we find.
• Failure on purpose. We're building collection programs for how world-model labs train, with freeform sessions that have no resets, task catalogs you can reweight mid-program, and failures and recoveries captured on purpose as well as when they happen on their own.
• More real floors. Factories, workshops and homes in additional countries, with the same scenes captured on UMI-style handheld grippers for teams whose robots use grippers.
• Independent audits. QA for data you already own or bought elsewhere, through the same gates our own data passes: automated rules, human adjudication, relabeling and re-syncing, plus human-verified gold labels to calibrate your own auto-labelers.

Before you sign your next collection contract, ask any vendor, us included, for three things:

1. Full episodes from a real work environment.
2. Failures and recoveries, labeled.
3. The QC report that shipped with the data.

Then tell us the task and the robot, and we'll show you ours.