A humanoid AI system has been developed that learns jointly from separately collected locomotion and object-manipulation data to perform multiple stages of a task in sequence.
A research team led by Professor Songhwai Oh of the Department of Electrical and Computer Engineering at Seoul National University College of Engineering has developed RISE (Responsibility-Induced Specialized Experts), a Vision-Language-Action (VLA) framework that learns jointly from locomotion and object-manipulation demonstrations collected independently and enables a humanoid robot to perform multi-step tasks sequentially. The team succeeded in naturally connecting locomotion and object manipulation without requiring additional demonstrations of multi-step tasks involving transitions between the two behaviors, and validated the framework in both simulation and experiments using a real humanoid robot, the Unitree G1.
The research was accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026), a leading international conference in robotics and intelligent systems.
In recent years, Vision-Language-Action (VLA) technology, which enables robots to jointly understand camera images and natural-language instructions and generate physical actions, has attracted growing attention in robotics. VLA is an AI model that allows a robot to understand its surroundings, interpret instructions such as “pick up the object and move it to the designated location,” and act accordingly.
Humanoid robots must perform multiple actions sequentially, such as walking, picking up objects, and moving them. In practice, however, locomotion and object manipulation involve substantially different joints and movement characteristics, making it difficult for a single AI model to learn both reliably. In particular, collecting demonstrations of multi-step tasks in which locomotion and manipulation alternate places a substantial burden on data collection because a failure at an intermediate stage can render the entire demonstration incomplete. Individual behaviors such as walking, picking, and placing, by contrast, can be collected relatively reliably. However, such data alone provide no information about when the robot should end its current behavior and transition to the next one, making it difficult to perform continuous tasks in real-world settings.
To address this problem, the research team developed RISE, an AI framework that allows locomotion and manipulation data to be collected separately while learning them jointly within a single model to perform multi-step tasks sequentially.
RISE combines a shared visual-language perception model with multiple action experts and a meta-controller that coordinates them. During training, the system evaluates how accurately each expert predicts the current action and calculates its “responsibility.” Experts with higher responsibility are designed to have a greater influence when learning from the corresponding data. As a result, without predefined roles, individual experts naturally specialize in locomotion or object manipulation.
The research team also used changes in the expert weights output by the meta-controller during execution as signals for task transitions. Building on the observation that expert weights vary according to the characteristics of the current behavior, the system determines when the current task has ended and naturally transitions to the next predefined task. This enables the robot to perform multi-step tasks in sequence using only demonstrations of individual behaviors collected separately, without the need to collect additional continuous demonstrations containing transitions between locomotion and object manipulation.
The research team validated RISE both in simulation and on a real Unitree G1 humanoid robot. The robot used in the experiments was equipped with two Dex3-1 robotic hands in addition to its basic whole-body joints, giving it a total of 43 controllable degrees of freedom (DoF).
In simulation experiments, the robot was assigned two- to four-stage tasks involving objects such as apples, bottles, and cups, including ▲ moving and then picking up an object, ▲ picking up an object and then moving and placing it down, and ▲ moving, picking up an object, moving again, and placing it down. Across all simulation experiments, RISE achieved an average success rate of 87.3%, outperforming the 73.6% achieved by a GR00T-based baseline with a single-expert architecture. GR00T is a general-purpose foundation model for humanoid robots released by NVIDIA.
In experiments using a real Unitree G1 robot, the system performed locomotion and manipulation tasks consisting of two stages, four stages, and up to five stages. The robot sequentially picked up an object and placed it at another location, then picked up a second object, moved to the target location, and placed it down. RISE achieved an average task success rate of 50.0%, 20 percentage points higher than the 30.0% recorded by the GR00T-based baseline. In particular, RISE performed tasks more reliably than the baseline as the number of task stages increased. These results confirmed that experts specializing in different behaviors can be trained using only separately collected locomotion and manipulation data, and that the robot can transition between behaviors at appropriate times to perform multi-stage humanoid tasks sequentially.
The significance of the research lies in proposing a way to reduce the burden of repeatedly collecting complete, continuous task demonstrations for humanoid robot training. The responsibility-based expert training approach also demonstrates the potential to expand toward integrating behaviors with different characteristics—including locomotion, object picking, bimanual manipulation, and tool use—within a single robot AI system.
The technology could provide a foundation for developing humanoids capable of performing complex tasks involving repeated locomotion and object manipulation. It could initially be applied to tasks in which locomotion and manipulation occur together, such as transporting goods in logistics centers and moving parts and tools in manufacturing environments. As adaptability to diverse environments and safety are further improved, its applications are expected to expand to areas such as transporting supplies in hospitals, assisting with caregiving, and organizing objects in homes.
Professor Oh, who supervised the research, said, “For humanoid robots to perform a variety of tasks in real-world environments, they must be able to reliably learn behaviors with different characteristics, such as locomotion and object manipulation, and transition naturally between them as needed.” He added, “This research is significant in demonstrating the possibility of performing continuous, complex tasks using separately collected locomotion and manipulation data, without having to directly collect demonstrations containing all of the different behaviors.”
He continued, “Going forward, we plan to move beyond the current structure, in which locomotion and manipulation occur sequentially, and expand the technology to more complex whole-body behaviors, such as manipulating objects while moving.”
Co-first author Hogun Kee earned his Ph.D. from the Robot Learning Lab at Seoul National University and is currently conducting physical AI research in industry. Co-first author Wooseok Oh plans to pursue a career as an AI and robotics researcher and developer at research institutes or companies in Korea or abroad after graduation. The IROS paper confirms Kee and Oh as equal contributors. RLLAB
This research was supported by grants funded by the Korean government (Ministry of Science and ICT, MSIT) through the Institute of Information & Communications Technology Planning & Evaluation (IITP) (No. RS-2024-00336738, “Development of Complex Task Planning Technologies for Autonomous Agents”) and the National Research Foundation of Korea (NRF) (No. RS-2026-25547694, “Heterogeneous Humanoids with Embodiment-Agnostic General Intelligence and Learning for Autonomous Execution of Complex Tasks”).
□ Introduction to the SNU College of Engineering
Seoul National University (SNU) founded in 1946 is the first national university in South Korea. The College of Engineering at SNU has worked tirelessly to achieve its goal of ‘fostering leaders for global industry and society.’ In 12 departments, 323 internationally recognized full-time professors lead the development of cutting-edge technology in South Korea and serving as a driving force for international development.
Experimental study
Not applicable
Learning Humanoid Loco-Manipulation with Responsibility-Induced Specialized Experts for Vision-Language-Action Models
The authors declare no competing interests.