Skip to main content

train

Module: GBC.gyms.isaaclab_45.workflows.rsl_rl.train

Overview 📖​

The main training script for reinforcement learning and imitation learning agents using the RSL-RL framework integrated with IsaacLab. This script provides a complete training pipeline that supports both standard RL training and reference-based imitation learning with multi-modal observations.

Core Features 🎯​

  • 🤖 RL/IL Training: Supports both reinforcement learning and imitation learning modes
  • 📊 Multi-modal Learning: Handles reference observations alongside standard environment observations
  • 🎥 Video Recording: Built-in video recording capabilities for training visualization
  • 🔄 Distributed Training: Multi-GPU and multi-node training support
  • 💾 Checkpoint Management: Automatic saving and loading of training checkpoints
  • 📋 Configuration Management: Hydra-based configuration system with CLI overrides

Command Line Usage 🚀​

Basic Training Command​

python train.py --task {TASK_NAME}

Advanced Training Options​

python train.py \
    --task {TASK_NAME} \
    --num_envs 4096 \
    --max_iterations 10000 \
    --seed 42 \
    --video \
    --video_interval 1000 \
    --distributed

Command Line Arguments ⚙️​

Core Arguments​

  • --task (str): Task name registered in GBC.gyms.isaaclab_45.lab_tasks.{YOUR_PROJECT_NAME}.__init__.py
  • --num_envs (int): Number of parallel environments to simulate
  • --max_iterations (int): Maximum training iterations to run
  • --seed (int): Random seed for reproducible training

Video Recording Arguments​

  • --video (flag): Enable video recording during training
  • --video_length (int, default=200): Length of recorded videos in steps
  • --video_interval (int, default=2000): Interval between video recordings

Training Mode Arguments​

  • --distributed (flag): Enable distributed training across multiple GPUs/nodes

Task Registration Requirements ⚠️​

Critical Registration Rule​

The --task parameter must exactly match the task name registered in:

GBC.gyms.isaaclab_45.lab_tasks.{YOUR_PROJECT_NAME}.__init__.py

Example Registration​

# In GBC.gyms.isaaclab_45.lab_tasks.humanoid_amp.__init__.py
import gymnasium as gym
from .humanoid_amp_env import HumanoidAmpEnv, HumanoidAmpEnvCfg

# Register the task
gym.register(
    id="Isaac-Humanoid-Amp-v0",  # ← This name must match --task argument
    entry_point="GBC.gyms.isaaclab_45.lab_tasks.humanoid_amp:HumanoidAmpEnv",
    kwargs={
        "env_cfg_entry_point": HumanoidAmpEnvCfg,
        "rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:HumanoidAmpPPORunnerCfg",
    },
)

Usage with Registered Task​

python train.py --task Isaac-Humanoid-Amp-v0

Main Function Pipeline 🔄​

Configuration Processing​

@hydra_task_config(args_cli.task, "rsl_rl_cfg_entry_point")
def main(env_cfg: ManagerBasedRLEnvCfg, agent_cfg: RslRlRefOnPolicyRunnerCfg):

Key Steps:

  1. 📝 CLI Override: Command line arguments override Hydra configurations
  2. 🎯 Environment Setup: Device assignment and seed configuration
  3. 🌐 Distributed Setup: Multi-GPU device and seed distribution

Environment Creation​

# Create IsaacLab environment
env = gym.make(args_cli.task, cfg=env_cfg, render_mode="rgb_array" if args_cli.video else None)

# Wrap for RSL-RL compatibility
env = RslRlReferenceVecEnvWrapper(env)

Training Execution​

# Create RSL-RL runner with multi-modal support
runner = OnPolicyRunnerMM(env, agent_cfg.to_dict(), log_dir=log_dir, device=agent_cfg.device)

# Execute training
runner.learn(num_learning_iterations=agent_cfg.max_iterations, init_at_random_ep_len=True)

Logging and Output 📁​

Directory Structure​

logs/rsl_rl/{experiment_name}/{timestamp}_{run_name}/
├── params/
│   ├── env.yaml          # Environment configuration
│   ├── agent.yaml        # Agent configuration
│   ├── env.pkl           # Pickled environment config
│   └── agent.pkl         # Pickled agent config
├── videos/train/         # Training videos (if enabled)
├── checkpoints/          # Model checkpoints
└── summaries/            # TensorBoard logs

Configuration Logging​

  • 🔧 Environment Config: Complete environment setup parameters
  • 🤖 Agent Config: RSL-RL algorithm configuration
  • 📊 Git State: Source code version information for reproducibility

Integration Components 🔗​

RSL-RL Integration​

  • OnPolicyRunnerMM: Multi-modal policy runner for reference-aware training
  • RslRlReferenceVecEnvWrapper: Environment wrapper for RSL-RL compatibility
  • RslRlRefOnPolicyRunnerCfg: Configuration class for reference-enabled training

Hydra Configuration System​

  • @hydra_task_config: Decorator for dynamic configuration loading
  • cli_args.update_rsl_rl_cfg: CLI argument integration with Hydra configs

Example Training Sessions 💡​

This section is generated by Claude, we recommend you to checkout for isaaclab running command line introductions here and replace the isaaclab.sh commands with the appropriate python train.py commands. We support all isaaclab command line arguments.

Standard RL Training​

# Train humanoid locomotion with pure RL
python train.py --task Isaac-Humanoid-v0 --num_envs 2048 --max_iterations 5000

Imitation Learning Training​

# Train with reference motion data
python train.py --task Isaac-Humanoid-Amp-v0 --num_envs 4096 --max_iterations 10000

Training with Video Recording​

# Record training videos every 500 iterations
python train.py --task Isaac-Humanoid-Amp-v0 --video --video_interval 500 --video_length 300

Distributed Training​

# Multi-GPU training
python train.py --task Isaac-Humanoid-Amp-v0 --distributed --num_envs 4096

Troubleshooting 🔧​

Common Issues​

Task Not Found Error​

# Error: gymnasium.error.UnregisteredEnv: No registered env with id: YourTask

Solution: Ensure task name matches registration in __init__.py

RSL-RL Version Compatibility​

# Error: RSL-RL version X.X.X is not supported

Solution: Install the GBC project's forked RSL-RL version

GPU Memory Issues​

# Reduce number of environments
python train.py --task YourTask --num_envs 1024  # Instead of 4096

Related Components 🔗​

  • 🏗️ Environment Configs: Task-specific environment configurations
  • 🤖 Agent Configs: RSL-RL algorithm and network configurations
  • 📊 Wrappers: Environment wrappers for RSL-RL integration
  • 💾 Checkpoint Utils: Model saving and loading utilities

This training script serves as the central entry point for all RL/IL training in the GBC framework, providing a robust and flexible training pipeline for humanoid robot learning tasks.