Research

Learning Parkour with Simulators (2024)

Demo Developers: Hikariu Wada, Momo Hanawa, Koki Fukuda, Yoshihiro Noumi

In the high-performance simulation environment “Isaac Gym,” reinforcement learning was applied to a quadruped robot, enabling it to acquire complex parkour-like movements. The advancements in simulation technology have made it possible to learn intricate behaviors in a short time and develop models that can adapt to real-world environments. This approach allows for the rapid collection of large amounts of training data, facilitating the creation of models capable of performing effectively in real-world scenarios.

REFERENCES

Cheng, Xuxin, et al. “Extreme parkour with legged robots.” 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024.

Robot Foundation Model Demo (2024)

Demo Developers: Yuya Ikeda, Aoi Horo, Hiroki Ishii, Yoshihiro Noumi

Using the robot foundation model “Octo,” we demonstrated multiple robots performing diverse tasks with a single model. Data combining camera images, joint angles, and instruction texts was collected and used for additional training of Octo. It was confirmed that the model could generalize and execute multiple tasks on xArm and Sawyer robots.

REFERENCES

Octo Model Team, Ghosh, D., et al. “Octo: An open-source generalist robot policy.” Proceedings of Robotics: Science and Systems, 2024.

Approach to GPSR Tasks and Their Solutions (2024)

Demo Developers: Aoi Horo, Hikaru Wada, Koki Fukuda, Yoshihiro Noumi

We used several foundation model technologies such as a large language model (GPT-4), a speech recognition model (Whisper), an object detection model (Detic), and a multimodal foundation model (CLIP). By integrating various foundation models and implementing them in a robot, it can comprehensively recognize the real world and generate appropriate actions based on its abilities in response to commands.

REFERENCES

基盤モデルを活用した自然言語による多様なタスク実現に向けたロボットシステムの統合

Self-Recovery Prompting: Promptable General Purpose Service Robot System with Foundation Models and Self-Recovery