How LeRobot is organized

5 min read · for SO-101 users

What this means for you: when you run lerobot-record, you're using the robot driver + teleop + dataset format. When you run lerobot-train, you're using the dataset + trainer + policy. Every other page on this site zooms into one of those five pieces.

  ┌─────────┐    ┌──────────┐    ┌──────────┐
  │ Teleop  │───▶│  Robot   │───▶│ Dataset  │
  │ (leader)│    │(follower)│    │ (record) │
  └─────────┘    └──────────┘    └────┬─────┘
                                      ▼
                                ┌──────────┐    ┌──────────┐
                                │  Policy  │◀───│ Trainer  │
                                │ (ACT…)   │    │ (train)  │
                                └──────────┘    └──────────┘

The basics

What is a policy?

A policy is the trained brain. You feed it what the robot sees (camera frames + joint angles) and it outputs what the robot should do next (target joint positions, or a short chunk of upcoming motions). ACT, Diffusion, SmolVLA, π0 are all different recipes for that brain.

What is a teleop?

A teleop (teleoperator) is whatever you use to drive the robot while recording demos. For SO-101 that's almost always a leader arm — a smaller twin you grip with your hand. The follower mirrors your motion. Other options: keyboard, gamepad, or a phone for 6-DoF (six degrees of freedom) tracking.

What's a robot driver?

The piece that talks to the actual motors over a USB or CAN bus. It reads joint angles and writes target positions every loop iteration. You don't usually touch it directly; the CLI scripts use it for you.

What's a LeRobotDataset?

A folder of recorded demos saved as Parquet tables (the numeric data: joint angles, actions, timestamps) plus MP4 videos (the camera frames), with a small JSON index that ties them together. It uploads cleanly to the Hugging Face Hub.

What ties it together?

The CLI scripts (lerobot-record, lerobot-train, lerobot-eval) and a set of config dataclasses — plain Python classes whose fields become CLI flags. Almost every flag you've ever passed is documented as one of those fields, so --help works on everything.

How you actually use it

The three CLI commands you'll hit most:

A typical SO-101 session: record 50 episodes → push the dataset to the Hub → fine-tune an ACT or Diffusion checkpoint on it → load the fine-tuned policy back and run lerobot-eval. All five Lego pieces, in order.

Things to know