Skip to main content

physics_modifier_manager

Module: GBC.gyms.isaaclab_45.managers.physics_modifier_manager

Overview 📖​

The PhysicsModifierManager is a specialized manager class that enables curriculum learning and progressive training in reinforcement learning and imitation learning environments. It allows users to introduce physics modifiers such as external forces, torques, random disturbances, and other physics interventions to gradually increase training difficulty or adapt agents to specific physical conditions.

This manager is particularly useful for:

  • 🎯 Curriculum Learning: Gradually increasing physics complexity during training
  • 🌍 Domain Adaptation: Adding realistic physics disturbances to improve robustness
  • 📈 Progressive Training: Starting with simplified physics and evolving towards realistic conditions
  • ⚙️ Environmental Conditioning: Training agents for specific physical scenarios

Core Features ⭐​

Dynamic Physics Modification 🔄​

  • ⚡ Real-time Updates: Physics modifiers are applied every physics step for immediate effect
  • 🎛️ Adaptive Parameters: Modifier parameters can be dynamically updated based on training progress
  • 🧠 Environment-aware: Modifiers can access environment state and respond intelligently
  • 📚 Curriculum Integration: Seamless integration with curriculum learning strategies

Flexible Configuration System 🛠️​

  • 🧩 Term-based Architecture: Each physics modifier is configured as a term with specific parameters
  • 🎨 Decorator Support: Uses @update decorator for sophisticated update strategies
  • 🔄 Hot-swapping: Modifier configurations can be changed during runtime
  • 📊 Hierarchical Override: Parameters can be overridden at multiple levels

Class Reference 📚​

PhysicsModifierManager 🏗️​

Inheritance: ManagerBase

The main manager class responsible for coordinating and applying physics modifiers.

Constructor 🔨​

def __init__(self, cfg: object, env: ManagerBasedEnv)

Parameters:

  • cfg: Configuration object containing physics modifier term definitions
  • env: The IsaacLab environment instance to apply modifiers to

Key Properties 🔑​

active_terms 📋​
@property
def active_terms(self) -> list[str]

Returns the list of currently active physics modifier term names.

Core Methods ⚙️​

apply() 🚀​
def apply(self, env_ids: Sequence[int] | None = None)

Applies all active physics modifiers to the specified environment instances.

Parameters:

  • env_ids: Environment IDs to apply modifiers to. If None, applies to all environments.

Behavior:

  • Executes every registered physics modifier function
  • Passes environment-specific parameters to each modifier
  • Handles parameter overrides and function-specific arguments
update() 🔄​
def update(self, env_ids: Sequence[int] | None = None)

Updates physics modifier parameters based on training progress and environmental conditions.

Parameters:

  • env_ids: Environment IDs to update modifiers for. If None, updates for all environments.

Features:

  • Called every observation step for adaptive parameter adjustment
  • Enables curriculum learning through progressive parameter modification
  • Supports complex update strategies through decorated functions
reset() 🔄​
def reset(self, env_ids: Sequence[int] | None = None) -> dict[str, float]

Resets physics modifiers for specified environments, typically called at episode boundaries.

Parameters:

  • env_ids: Environment IDs to reset modifiers for. If None, resets for all environments.

Returns:

  • Dictionary of reset-related metrics (currently empty)

Configuration Management 🛠️​

set_term_cfg() ⚙️​
def set_term_cfg(self, term_name: str, cfg: PhysicsModifierTermCfg)

Dynamically updates configuration for a specific physics modifier term.

Parameters:

  • term_name: Name of the term to update
  • cfg: New configuration for the term
get_term_cfg() 📄​
def get_term_cfg(self, term_name: str) -> PhysicsModifierTermCfg

Retrieves current configuration for a specific physics modifier term.

Parameters:

  • term_name: Name of the term to retrieve configuration for

Returns:

  • Current PhysicsModifierTermCfg for the specified term

Physics Modifier Function Architecture 🏛️​

Function Decorator System 🎨​

Physics modifier functions must be decorated with the @update decorator from physics_modifier_function_wrapper.py:

from GBC.gyms.isaaclab_45.managers.physics_modifier_function_wrapper import update

@update(update_strategy=custom_update_strategy)
def my_physics_modifier(env, env_ids, **params):
    # Modifier implementation
    pass

Update Strategy Pattern 📋​

The @update decorator supports sophisticated parameter update strategies:

def scaling_modifier(current_overrides: dict, new_kwargs: dict) -> dict:
    """Example update strategy that scales parameters"""
    updated = dict(current_overrides)
    if new_kwargs:
        for k, v in new_kwargs.items():
            if isinstance(v, (int, float)):
                updated[k] = v * 0.6  # Scale down by 40%
    return updated

Practical Example: External Force for Curriculum Learning 🎓⚡​

Use Case: Baby Walker Training 👶🚶​

The following example demonstrates a sophisticated physics modifier for robot locomotion training:

class ExternalForceUpdater:
    """Adaptive update strategy for external force curriculum"""
    
    def __init__(self):
        self.history_base_contact = deque(maxlen=320)
        self.last_update_cnt = 0

    def update_external_z_force_base(self, current_overrides: dict, new_kwargs: dict) -> dict:
        env = new_kwargs["env"]
        
        # Monitor base contact performance
        base_contact = env.unwrapped.extras["log"]["Episode_Termination/base_contact"]
        self.add_base_contact_val(base_contact)
        base_contact_avg = self.get_base_contact_avg()
        
        # Adaptive parameter adjustment
        if base_contact_avg < new_kwargs["update_threshold"]:
            # Robot is struggling, make training easier
            current_overrides["apply_offset_range"] *= new_kwargs["update_ratio"]
        else:
            # Robot is succeeding, increase difficulty
            current_overrides["apply_offset_range"] /= new_kwargs["update_ratio"]
            
        return current_overrides

@update(update_strategy=ExternalForceUpdater().update_external_z_force_base)
def external_z_force_base(
    env: ManagerBasedRLEnv, 
    env_ids: torch.Tensor,
    max_force: float,
    apply_offset_range: float,
    apply_force_duration_ratio: float,
    asset_cfg: SceneEntityCfg = SceneEntityCfg("robot"),
    working_mode: str = "spring",
    ref_height_offset: float = 0.15,
    update_threshold: float = 1.1,
    update_ratio: float = 1.05,
    update_interval: int = 320,
    apply_offset_range_bound: tuple[float] = (0.01, 1.05),
):
    """
    Applies adaptive external z-direction force to robot base for curriculum learning.
    
    This modifier implements a "baby walker" training strategy where:
    - External upward force prevents robot from falling
    - Force magnitude and application range adapt based on performance
    - Two working modes: "spring" (proportional) and "constant" (binary)
    
    Working Modes:
    - Spring: F = -k * (z - ref_height) with clamping
    - Constant: F = max_force when height deviation exceeds threshold
    
    Curriculum Progression:
    - Successful training reduces force magnitude and range
    - Poor performance increases assistance
    - Force duration can be shortened as training progresses
    """
    
    # Implementation details for force calculation and application
    # ... (force computation logic)
    
    # Apply force through IsaacLab's external force system
    asset.set_external_force_and_torque(
        forces=forces, 
        torques=torque, 
        body_ids=asset_cfg.body_ids, 
        env_ids=env_ids
    )

Configuration Registration ⚙️​

@configclass
class PhysicsModifiersCfg:
    """Physics modifiers configuration"""
    
    external_z_force_base = PhysicsModifierTermCfg(
        func=external_z_force_base,
        params={
            "max_force": 1500.0,
            "apply_offset_range": 0.15,
            "apply_force_duration_ratio": 1.0,
            "asset_cfg": SceneEntityCfg("robot", body_names=["torso_link"]),
            "update_threshold": 1.1,
            "update_ratio": 1.05,
            "update_interval": 320,
            "apply_offset_range_bound": (0.01, 50.0),
        },
        description="Adaptive external z-force for curriculum learning (baby walker)",
    )

Integration with Environment 🌐​

Environment Configuration 🔧​

@configclass
class MyEnvCfg(ManagerBasedRLEnvCfg):
    # ... other configurations
    
    physics_modifiers: PhysicsModifiersCfg = PhysicsModifiersCfg()

Runtime Usage 🏃‍♂️​

# In environment step loop
env.physics_modifier_manager.apply()  # Apply modifiers every physics step
env.physics_modifier_manager.update() # Update parameters every observation step

# Dynamic configuration changes
new_cfg = PhysicsModifierTermCfg(
    func=external_z_force_base,
    params={"max_force": 2000.0}  # Increased force
)
env.physics_modifier_manager.set_term_cfg("external_z_force_base", new_cfg)

Advanced Features 🚀​

Multi-modal Training Strategies 🎯​

Physics modifiers can implement complex training curricula:

  1. 📈 Progressive Difficulty: Start with high assistance, gradually reduce
  2. 📊 Performance-based Adaptation: Adjust based on success metrics
  3. 🔄 Multi-phase Training: Different modifiers for different training stages
  4. 🌍 Environmental Conditioning: Specific modifiers for target deployment conditions

Performance Considerations ⚡​

  • 🔧 Efficient Updates: Update strategies should be computationally lightweight
  • 🎯 Selective Application: Use env_ids parameter for targeted modification
  • 💾 Memory Management: Use circular buffers for historical data
  • 📏 Parameter Bounds: Always implement reasonable parameter bounds

Best Practices 💡​

Function Design 🎨​

  1. 📊 Stateful Updates: Use class-based update strategies for complex curricula
  2. 🛡️ Robust Bounds: Always clamp parameters to safe ranges
  3. 📈 Performance Monitoring: Track relevant metrics for adaptive updates
  4. 🔄 Graceful Degradation: Handle edge cases and parameter edge values

Curriculum Design 📚​

  1. 🌱 Start Simple: Begin with high assistance/simplified physics
  2. 📊 Monitor Progress: Use performance metrics to guide curriculum progression
  3. 🌊 Smooth Transitions: Avoid abrupt parameter changes that could destabilize training
  4. 🎯 Target-oriented: Design curriculum with final deployment conditions in mind

Integration Patterns 🔧​

  1. 🧩 Modular Design: Keep physics modifiers independent and composable
  2. ⚙️ Configuration Flexibility: Support both static and dynamic parameter updates
  3. 📝 Logging Integration: Track modifier parameters and effects for analysis
  4. 🌐 Environment Compatibility: Ensure modifiers work across different environment configurations

Error Handling 🛡️​

The manager includes robust error handling:

  • ✅ Type Validation: Ensures all terms are properly configured
  • 🔍 Function Validation: Verifies required update attribute on modifier functions
  • 📋 Parameter Validation: Checks function signatures and parameter compatibility
  • 🔒 Runtime Safety: Handles edge cases in parameter updates and force applications

Related Components 🔗​

  • 📄 PhysicsModifierTermCfg: Configuration class for individual physics modifier terms
  • 🎨 physics_modifier_function_wrapper: Decorator system for update strategies
  • 🏗️ ManagerBase: Base class providing core manager functionality
  • 🌐 IsaacLab Integration: Seamless integration with IsaacLab's physics simulation system