Abstract

The combination of imitation learning and reinforcement learning have achieved remarkable results in the control of whole-body motions of humanoid robots, enabling robots to perform a wide range of human-like motions. However, for motions that cannot maintain self-stabilizing, these methods are often affected by various factors and did not achieve satisfactory results. When analyzing non-self-stabilizing motions such as sitting down and lying down, we found that there is usually a phase of weightlessness during which the human body's lower-limb muscles relax and the body falls weightlessly, with only the upper body stability being controlled mainly by the waist. Referring to this characteristic of human motion, we propose a method that involves the weightlessness mechanism, which is essentially a joint torque output-state machine for imitation learning. Based on imitation learning, we add a state machine strategy to the joints of the humanoid robot to determine whether the joints should output the control torques. While maintaining the stability and movement of the upper limbs, this method determines whether the robot's lower limbs enter a state of weightlessness. Taking the action of sitting on chairs with different heights as an example, we have implemented a more natural chair-sitting motion using the Unitree G1 robot in both simulation and real-world scenarios, thereby demonstrating the effectiveness of the method we propose.

Framework

(a) Collect sitting motion datasets, apply motion retargeting, use sampling smoothing for low-quality data, and filter for imitable actions. (b) Label the robot's weightless state during data processing and train a weightlessness state policy network using an LSTM to determine weightless state based on historical and future pose trends. (c) Learn sitting action policy via imitation learning. (d) Integrate weightlessness policy network into imitation learning to fine-tune action policy, outputting zero torque for lower-limb joints in weightless state during environment updates. (e)(f) Validate action policy in MuJoCo environment and deploy algorithm on real robot.

Experiments