I Built an SO-100 and Tried to Make It Smart: LeRobot, π0, and GR00T N1.6-3B

Where this started

I did not want a servo toy that can replay a few hard-coded poses. I wanted something I could:

• teach from demonstrations;
• measure with real metrics (accuracy, repeatability, success rate);
• iterate on without rewriting everything.

In my thesis, I built an SO-100 prototype and ran VLA-style control with π0 (through LeRobot/OpenPI). Now I am planning the next step: GR00T N1.6-3B, without breaking the parts that already work.

Why LeRobot

I picked Hugging Face LeRobot because it is process-first. The docs make the robotics loop explicit:

• calibrate;
• record datasets into LeRobotDataset;
• train a policy;
• rollout and evaluate.

That matters because in real robotics, random scripts kill more time than model choices.

Model layer: what I take from GR00T N1.6-3B

My target foundation policy is nvidia/GR00T-N1.6-3B. Translated into engineering terms:

• inputs: images + language instruction + proprioception;
• outputs: continuous actions;
• action head uses flow matching (diffusion family) and predicts action chunks.

A practical note: LeRobot has a well-documented integration path for GR00T N1.5 (docs). I treat that as a stable baseline. N1.6-3B is the next step, but only after my dataset and control loop are solid.

The reality of SO-100 (numbers from my thesis)

These ranges define what is realistic on a low-cost 3D-printed arm:

• positioning accuracy: ~8–12 mm;
• repeatability: ~2–4 mm;
• basic task success: 65–93%;
• control frequency on a laptop (RTX 3060 Laptop): ~5–7 Hz.

This is not bad. It is honest. And it makes it clear which tasks are worth attempting.

Step-by-step: the pipeline I actually use

I am intentionally writing this as something you can copy and adapt. Where possible, I use the official LeRobot commands from the documentation.

Step 0. One-time setup

1. Log into the Hugging Face CLI so datasets can be pushed to the Hub.
2. Pick stable robot.id and teleop.id and keep them consistent across sessions.

huggingface-cli login --token ${HUGGINGFACE_TOKEN} --add-to-git-credential
HF_USER=$(huggingface-cli whoami | head -n 1)

Step 1. Install LeRobot (and Feetech)

SO-100 docs require the Feetech extras:

pip install -e ".[feetech]"

If you are targeting GR00T, there is a dedicated install section for lerobot[groot] and flash-attn (CUDA-only): https://huggingface.co/docs/lerobot/main/groot

Step 2. Find ports and configure motors

I learned to do this before final assembly. It is painful to access motor connectors later.

Commands from SO-100 docs:

lerobot-find-port
 
lerobot-setup-motors \
  --robot.type=so100_follower \
  --robot.port=/dev/tty.usbmodem585A0076841
 
lerobot-setup-motors \
  --teleop.type=so100_leader \
  --teleop.port=/dev/tty.usbmodem575E0031751

Step 3. Calibration (the non-negotiable ritual)

Calibration is the difference between a clean dataset and a week of confusion.

From SO-100 docs:

lerobot-calibrate \
  --robot.type=so100_follower \
  --robot.port=/dev/tty.usbmodem58760431551 \
  --robot.id=my_follower
 
lerobot-calibrate \
  --teleop.type=so100_leader \
  --teleop.port=/dev/tty.usbmodem58760431551 \
  --teleop.id=my_leader

My rule: I keep the same calibration poses every time and treat it like a protocol.

Step 4. Record demonstrations

LeRobot tutorials show the flow on SO-101, but the logic is the same: follower + leader + cameras + --dataset.single_task.

python -m lerobot.record \
  --robot.type=so100_follower \
  --robot.port=/dev/tty.usbmodem585A0076841 \
  --robot.id=my_follower \
  --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 1280, height: 720, fps: 30}}" \
  --teleop.type=so100_leader \
  --teleop.port=/dev/tty.usbmodem58760431551 \
  --teleop.id=my_leader \
  --display_data=true \
  --dataset.repo_id=${HF_USER}/so100_pickplace_v1 \
  --dataset.num_episodes=50 \
  --dataset.single_task="Pick the block and place it into the bin"

Thesis lesson: 50 clean episodes in stable conditions beat 10 long, cinematic, noisy demos.

Step 5. Replay as a sanity check

This is cheaper than training.

python -m lerobot.replay \
  --robot.type=so100_follower \
  --robot.port=/dev/tty.usbmodem585A0076841 \
  --robot.id=my_follower \
  --dataset.repo_id=${HF_USER}/so100_pickplace_v1 \
  --dataset.episode=0

If replay is bad: it is almost always calibration, mechanics, or data. Not the model.

Step 6. Train a baseline policy

I like starting with policy.type=act because it is a fast baseline and instantly tells you if the dataset makes sense.

python lerobot/scripts/train.py \
  --dataset.repo_id=${HF_USER}/so100_pickplace_v1 \
  --policy.type=act \
  --output_dir=outputs/train/act_so100_pickplace_v1 \
  --job_name=act_so100_pickplace_v1 \
  --policy.device=cuda \
  --wandb.enable=true

Step 7. Where GR00T N1.6-3B fits

My migration logic:

1. Pipeline is stable.
2. Dataset is repeatable.
3. Baseline policy behaves.
4. Only then I swap policy layers for GR00T.

LeRobot docs provide the policy.type=groot path for GR00T N1.5; I treat that as a verified checkpoint. Then I move toward N1.6-3B.

For the short operational checklist (no story): SO-100 + GR00T N1.6: my step-by-step playbook.

Three things I wish I did earlier

1. A strict pre-flight checklist (no "quick run" without it).
2. Thermal logging every session.
3. Separate modes: record, replay, eval.

Why this is worth doing

For me the point is not "a cool arm video". It is a repeatable loop: data, training, rollout, iteration. Once the loop is stable, you stop guessing and you start improving.

Final takeaway

LeRobot is not "just a library". It forces an engineering workflow: stable IDs, calibration discipline, data validation, then models.

SO-100 is a great lab platform precisely because it is unforgiving: play, jitter, thermals, latency. Changing the policy does not make those problems disappear. You fix them with process and hardware, then you earn GR00T.