Research project · Build log
A robot arm that learns to turn poundo yam from videos of people making it. Iyan is Yoruba for pounded yam.
Started 19 Oct 2026 · Base arm: FR3 · Next: understanding the task
Pounded yam is a smooth, stretchy West African staple, eaten with soups like egusi. It was made by pounding boiled yam in a wooden mortar.
Today most people make poundo from yam flour instead. The flour is stirred into hot water in a pot.
Then it’s turned hard with a wooden stick until smooth. The dough starts loose and ends thick and sticky, so the resistance keeps changing. Hot, heavy, repetitive work.
Photos: HarunaSylvester; Blackcoffeegirl; Koutchika Lihouenou Gaspard; all CC BY-SA 4.0, cropped; via Wikimedia Commons. 02 and 03 show elubo and amala, poundo’s yam-flour cousin; the stir and the turn are the same.
Can I teach a robot to make food from video while incorporating uncertainties like poundo’s properties, all without a force sensor, for a simpler product?
The motion comes from clips of people turning poundo. The stick-tip path is tracked and retargeted to a robot arm.
The dough goes from loose flour and water to a thick, sticky mass, and no two pots behave the same. The robot has to adapt as the resistance changes.
The policy never reads force. A load-cell stick measures what a real turn takes, and the FR3’s joint torque sensors are used only to check the policy’s work, never as its input.
Poundo yam is a West African staple and my favourite food. You stir yam flour into hot water, then keep turning it hard with a wooden stick until it's smooth and stretchy. It's tedious, and it takes real effort.
For a robot it's a hard task. The dough starts as loose flour and water and turns into a thick, sticky mass, so the stick is pushing through something whose resistance keeps changing. The pot wants to move. And the only "instructions" that exist are videos of people doing it, which show how the stick moves but not how hard it's being pushed.
Design an arm and a first gripper that turn a pot of poundo from loose flour to smooth, lump-free swallow, with the motion learned from videos of people turning it.
It's also practice in building efficiently: one project, documented as it goes, failures included.
Motion from video. Clips of people turning poundo are tracked for the stick-tip path, then retargeted to the arm with IK.
Force from a sensor. Video can't show force, so a load-cell stick logs what a real turn takes.
Franka Research 3 as the base, with a gripper on its ISO 9409-1-50-4-M6 flange holding a plain wooden stick.
Simulation first, in MuJoCo, before anything touches a real pot.
Tools installed on Windows; arm sim runs locally (SO-101 first, now FR3).
Stages of turning broken down; 30+ clips collected and curated.
Instrumented stick built; peak force and torque logged across stages.
Requirements table set from measured loads.
v1 stick gripper on the FR3 flange; pot fixture; torque sheet.
Stick-tip trajectories extracted and retargeted to the arm in sim.
A policy or scripted mapping replays turning strokes in MuJoCo.
Sim beside the original clip; optional real pot test.
A short deck and a demo video.
Replaced SO-101 with the FR3: 3 kg payload, the biggest MuJoCo and robot-learning ecosystem, and seven joint torque sensors to check a policy that never reads force. UR5e was the backup. Opened the FR3 v2 model from MuJoCo Menagerie in the MuJoCo viewer on my own machine. The tool flange is ISO 9409-1-50-4-M6: four M6 bolts on a 50 mm circle, with a Ø6 H7 dowel. Next: break turning into stages and start collecting clips.
Cloned SO-ARM100 and opened the SO-101 model in the MuJoCo viewer on my own machine. That closes P0. Next: break turning into stages and start collecting clips.
Git, Cursor and Miniforge installed; a conda env with Python 3.12 and ffmpeg; LeRobot, MuJoCo and FiftyOne installed and importing. The SO-101 MuJoCo model loads and simulates on MuJoCo 3.16.0, with 6 joints and 6 actuators.
Picked the project, wrote the brief, and compared open base arms: SO-101 (Apache-2.0), AR4 MK5 (non-commercial licence), PAROL6 (paid STEP). Chose SO-101 and decided to stay on Windows. Set up the engineering log and a task board. Learned: video only gives kinematics, so force has to come from my own measurements, and open-hardware licences differ a lot.
Franka Research 3 by Franka Robotics, the base arm. The FR3 model is derived from Franka’s franka_description.
MuJoCo Menagerie franka_fr3_v2 by Google DeepMind, released under the Apache-2.0 licence. The FR3 MuJoCo model comes from here.
SO-ARM100 / SO-101 by The Robot Studio, released under the Apache-2.0 licence. The first base arm and its MuJoCo model, used until 21 Oct 2026.
MuJoCo, the physics simulator the arm runs in.
LeRobot by Hugging Face, for robot learning.
FiftyOne by Voxel51, for curating the video clips.
Poundo photos from Wikimedia Commons, all under CC BY-SA 4.0, cropped: pounding boiled yam by HarunaSylvester; a woman selling elubo (yam flour) in Ipata market, Ilorin by Blackcoffeegirl; making of amala in Badagry, Lagos by Koutchika Lihouenou Gaspard.