LeRobot: Ready for LeRobot.
Annotated demonstrations in LeRobot format, with the dataset layout agreed with you beforehand.
- LeRobot dataset delivery
- Task episodes and aligned annotations
- Fields agreed with your team
Hand and object annotation
QpiAI Pro annotates first person video for robotics.
You record people doing a task. Pro labels what their hands do. Each recording gets hand joints, the objects in use, task steps and estimated 3D positions. You can check every label against the video, and we deliver the data in a standard format or a custom format.
See a sample runThe seven stages of annotation.
Every recording goes through the same seven stages. Pick one to see what it does and what it adds to the data.
Prepare the recording
Split the recording into frames, each with its timestamp, and keep the original timing. Note the scene and the task, so later stages know what to look for.
Find and follow the objects in use
Find the objects the hands touch and follow each one from frame to frame, with its own ID and outline. If a track breaks or two objects swap IDs, it can be joined back together.
Track both hands and their joints
Find 21 joints on each hand, keep left and right apart and smooth the motion over time. Frames where a hand was missing or had to be filled in are flagged for review.
Work out which hand holds what
Give each tracked object a name, then work out which hand touches which object, and from when to when.
Estimate motion in 3D
Use the camera's movement and the depth of the scene to place hand joints, grasp points and held objects in 3D.
Split into steps and review
Check that the tracks are complete and consistent, then split the clip into task steps. Each step gets start and end timestamps, a short description, its main hand and the objects used. A person then checks everything against the video and fixes mistakes before release.
Deliver the dataset
Package the approved data with step details and a description of every field, in a standard format or a custom format.
Recorded hand joints and episode paths
Both hands on the chocolate-box recording, with their joints and the path each one traces during the current step. Pick a hand, drag the timeline or jump to another step.
The sample shows a first-person recording of someone unwrapping a gift box and the task steps the pipeline found in it.
Watch the original sample recordingDelivered in your format.
We deliver the annotations as a LeRobot dataset, as ROS 2 bags or in your own schema, with the fields, names and file layout your team already uses.
ROS 2 bag: Replay it in ROS 2.
One ROS 2 bag per task step, stored as MCAP with standard message types only. Play it with ros2 bag play, or open it in RViz or Foxglove, with nothing extra to install.
- One MCAP bag per task episode
- Hand joints, wrists, grasp points and objects on /tf and standard topics
- Tested with ROS 2 Humble and Jazzy
Custom format: Matched to your schema.
Keep the schema your team already uses. We match its fields, names and file layout.
- Your schema and required fields
- Your names and folder layout
- Files that load in your tools
Sample dataset
This recording was delivered as Parquet tables. Browse the fields or see every file.
This sample uses metres in the camera frame: x right, y down, z forward.
Common questions
What is egocentric video annotation?
It means labelling video filmed from the wearer's point of view. QpiAI Pro marks the hand joints, the objects being handled and the task steps, and estimates where the hands and objects are in 3D.
What annotations does QpiAI Pro provide?
Annotations include 21 joints per hand, left and right hand identity, object tracks and names, hand–object interactions, step timestamps, grasp points and estimated 3D positions. The sample on this page shows them next to the original video.
Do you support LeRobot dataset format?
Yes. We deliver in LeRobot format and agree the dataset layout and fields with you first.
Can you deliver ROS 2 bags?
Yes. Each task step comes as its own ROS 2 bag in MCAP format, using only standard message types, so it plays with ros2 bag play and opens in RViz or Foxglove. Hand joints, wrists, grasp points and object poses are in the camera frame, in metres.
Can you deliver data in our own format?
Yes. Send us your schema, required fields, naming and folder layout, and we will deliver in that format.
Do recordings require gloves or motion-capture markers?
No. The pipeline works from ordinary first-person video, with no markers, gloves or extra sensors. A person reviews the annotations before release, and you keep the original video to check them against.
See it on your footage.
Building robot learning or embodied AI? Send us a short demo clip from your team's first-person cameras. We will annotate it and send back the same outputs you see on this page, so you can judge the quality on your own data.
Back to the pipeline