Skip to content

Latest commit

 

History

History
32 lines (25 loc) · 3.79 KB

File metadata and controls

32 lines (25 loc) · 3.79 KB

Debugging and Challenges Log

This log documents key challenges encountered during the implementation and execution of the RIPL prospective assignment, following our research philosophy of transparency and rigorous scientific methodology.


Challenge 1: Diffusion Policy training instability and visual overfitting

  • Observed: During training of baseline Diffusion Policy (T-I), training loss decreased steadily, but evaluation success rates remained low (~10-15%) even on nominal states. In-distribution rollouts showed the robot end-effector missing the T-block completely or clipping its edges.
  • Hypothesis: The visual encoder (ResNet18 backbone) is overfitting to color textures and minor camera noise, failing to extract robust spatial features required for precise policy actions.
  • Experiment: Tested adding spatial softmax layers after the visual feature maps and applied random cropping/light color jitter data augmentation on visual inputs during training.
  • Resolution: Adding a Spatial Softmax layer reduced the latent representation dimension to coordinate-like keypoints, improving spatial generalization. Incorporating random cropping stabilized validation success rate, bringing nominal baseline success to 85.3%.
  • Lesson: High-capacity visual backbones (like ResNet) easily overfit to demonstration backgrounds in simulator environments. Projecting features to low-dimensional keypoints is critical for action-based policies.

Challenge 2: LLM Reward Hacking (Geometric distance exploit)

  • Observed: The initial reward function generated by the LLM in T-III for Failure A (Corner Trapping) produced high rewards during PPO training, but the physical robot policy did not recover. Instead, the pusher was observed sliding repeatedly along the top flat bar of the T-block without actually pushing it.
  • Hypothesis: The LLM prioritized minimizing the geometric Euclidean distance between the pusher end-effector and the block center (ee_block_distance). The PPO agent found a shortcut to maximize this reward term by maintaining static contact without doing any actual work.
  • Experiment: Modified the prompt template in T-III to require a validation step, explicitly defining reward hacking. Instructed the LLM to balance contact incentives with progress-based reward terms, penalizing static velocity when far from the goal.
  • Resolution: Refined LLM prompt to include temporal state variables (prev_state) to measure task velocity. The updated reward penalized steps where the pusher contact was active but the block velocity was near zero.
  • Lesson: Dense distance-based rewards are highly vulnerable to local minima. Combining state-based distance limits with temporal velocity terms prevents static exploits.

Challenge 3: Catastrophic Forgetting during Residual Policy RL

  • Observed: Early PPO fine-tuning of the residual policy on Failure A led to rapid improvement on targeted failure states, but nominal evaluation success dropped from 85.3% to ~42.0%.
  • Hypothesis: Training the residual policy exclusively on failure initial states (P_failure) caused the residual MLP to output non-zero correction forces even during normal/nominal states, disrupting the base policy.
  • Experiment: Implemented two changes:
    1. A mixed training distribution: P_train = 0.3 · P_nominal + 0.7 · P_failure.
    2. A learned gating network g(s) ∈ [0, 1] that scales the residual action.
  • Resolution: The learned gate effectively mapped to near-zero in nominal states, leaving the base Diffusion Policy actions unmodified. The nominal success rate was preserved at 84.7% (only a 0.6% drop).
  • Lesson: Residual policies must be localized. Without a gating mechanism or mixed training distribution, the policy correction generalizes globally and disrupts working nominal behaviors.