Eithelmir
Eithelmir is a marketplace for the human demonstration data that humanoid robots train on. Labs post a task they need recorded and fund it; people record it with the glasses, watch and phone they already own. I built the pose pipeline underneath it and the web infrastructure around it.
- ROLE
- Founder
- WHEN
- May 2026 - Present
- WHERE
- Vancouver, BC
Why wearables instead of a capture suit
Motion capture that works already exists, and it lives in a room with markers on the walls and a suit you have to be fitted into. That constraint is the reason there is so little of this data: nobody can record someone folding laundry in their own kitchen if the recording requires a lab. Eithelmir's bet is that the sensors are already part of the outfit. A pair of Ray-Ban Meta glasses gives an egocentric camera at roughly eye height, and an IMU on each wrist tracks the hands through everything that camera cannot see. Neither is a motion capture rig on its own. The 9.8 cm figure is the answer to whether the pair is.
Getting the clocks to agree
The hard part turned out not to be the model, it was time. Independent consumer devices drift against each other over a session, and a pose estimate assembled from streams that disagree by 80 ms is worse than one built from video alone. The fix is deliberately low-tech. Every session opens with a clap, whose transient is sharp in both wrist accelerometers and loud in the audio track, so a single event lines every stream up to the frame.
Marking what was measured and what was guessed
A pose pipeline produces a number for every joint whether or not it had any evidence for that joint. If both hands leave the camera frame, the model still returns hand positions, and they still look plausible. That is the failure mode that makes a dataset worthless to a lab, because they cannot tell which frames to trust. So every field that ships carries a flag saying whether it was measured or inferred, and every contact event records whether it came from IMU alone or from IMU and video together. It makes the numbers look worse and the dataset actually usable.
Contributions at a glance
- Built a multi-modal pipeline that predicts half-body pose from egocentric video and two wrist IMUs, reaching 9.8 cm mean joint error against multi-view RGB triangulation
- Designed a development interface for visualising 3D limb localisation and reconstruction, combining SMPL avatars, MANO hand rigs and controlled recording sessions
- Built and shipped app.eithelmir.com: the bounty board, the contributor wallet, and the upload API the iOS capture apps post to, along with the payment handling behind it