Isaac Lab Locomotion Training

Training quadruped and humanoid locomotion policies with deep reinforcement learning.

Legged robot locomotion training in Isaac Lab
Google Drive Isaac Lab Documentation

Story

I started this project to understand how locomotion policies are built rather than simply running Isaac Lab examples. I focused on how observations, rewards, and curricula shape robot behavior.

I deployed Isaac Lab on an AWS EC2 instance with an NVIDIA A10G. The setup required resolving driver and Vulkan issues, running Isaac Sim headlessly, and streaming it through WebRTC.

I then trained Ant, Anymal-C, and H1 policies. Ant and H1 learned useful motion, while Anymal-C exposed how a policy can improve its reward without learning the intended gait.

Design

Infrastructure & Simulation

I ran Isaac Lab on Ubuntu using an AWS EC2 instance with an NVIDIA A10G. Building a usable remote simulator required matching the NVIDIA driver, fixing Vulkan initialization, and separating rendering failures from training failures.

The final setup combined headless training, WebRTC, TensorBoard, and tmux so long runs could be monitored and recovered remotely.

Reinforcement Learning Pipeline

I used RSL-RL's PPO implementation with PyTorch to train many environments in parallel. For each robot, I selected the observations, joint actions, commands, and termination conditions that defined its MDP.

A terrain curriculum increased difficulty as the policy became more reliable. TensorBoard tracked individual reward terms and episode statistics throughout training.

Reward Engineering

Anymal-C's reward increased from 0.02 to 3.29 and its average episode length grew from 33 to 679 steps, but it learned to balance instead of walking forward.

Stability and survival outweighed velocity tracking, making standing still a useful solution. H1 began walking within a few hundred iterations, reinforcing the need to judge reward curves alongside the robot's actual behavior.

Results

  • Trained the Ant policy to produce stable locomotion.
  • Trained Anymal-C for 19,000 iterations; reward rose from 0.02 to 3.29 and episode length from 33 to 679 steps, revealing a reward design that favored balance over walking.
  • Produced walking behavior on the H1 humanoid within the first few hundred iterations.

Technologies

  • Isaac Lab
  • Isaac Sim
  • RSL-RL
  • PyTorch
  • AWS EC2
  • Ubuntu
  • NVIDIA Vulkan
  • WebRTC
  • TensorBoard
  • Python
  • tmux