Robots Learn to Execute Commands with New AI System

In a groundbreaking development within the field of technology, a novel artificial intelligence (AI) system has been unveiled that empowers robots to understand and promptly execute commands given by humans. This innovative system integrates language models with software designed for robotic control, facilitating the transformation of both spoken and written instructions into specific actions. This advancement has the potential to fundamentally alter approaches to automation.

According to a report by Interesting Engineering, a publication that highlights achievements in robotics and artificial intelligence, researchers from Huawei Noah’s Ark Lab, the Technical University of Darmstadt, and ETH Zurich have collaborated to create a system that combines large language models with the Robot Operating System (ROS). This integration enables robots not only to perceive instructions but also to carry them out in physical environments, marking a significant stride in the evolution of autonomous technologies.

The system developed under this project processes textual commands and breaks them down into sequential steps. For instance, the command to lift a green block and place it on a black shelf is transformed into a series of actions that the robot performs through ROS. This process is crucial as it significantly simplifies the interaction between humans and machines.

Researchers emphasize that creating autonomous robots capable of translating natural language into reliable physical actions remains a critical challenge for artificial intelligence. They have demonstrated that the combination of a language model with ROS creates a universal system for 'embodied' intelligence, and the full implementation of this system has been made publicly available, allowing other researchers to utilize and enhance this technology.

The platform developed as part of this project merges the analytical capabilities of language models with a popular robotic control system. This allows for the interpretation of instructions without the need for separate programming for each task, greatly simplifying the robot training process.

The authors of the project explain that the agent automatically converts the outputs of the language model into robot actions, supports various execution modes, can learn new basic skills through imitation, and refine them based on feedback from humans or the environment. This opens up new horizons for the application of robots across various sectors, from industry to household tasks.

The system features two methods for task execution. In the first case, the model generates small code snippets to control the robot. In the second, it constructs decision structures known as behavior trees, which help adapt if a particular step fails. This enhances the system's flexibility and allows it to handle both simple and complex tasks, which is a vital aspect of robotics.

During testing on various robots, the system was able to reliably interpret commands and perform actions in diverse scenarios. According to the authors, experiments demonstrated the robustness, scalability, and versatility of the approach, particularly for prolonged tasks, object rearrangement on tables, action optimization, and remote control. This indicates that the new technology could find widespread application across different fields.

All results were obtained using pre-trained open-source language models. The robots can also learn based on feedback and gradually improve their performance without complex reprogramming. This represents another significant achievement that could greatly simplify the process of integrating robots into everyday life.

Researchers believe that the combination of language understanding with physical actions could accelerate the deployment of robots in dynamic environments where adaptability is crucial. In the future, they plan to scale the system for more complex tasks and various types of robots, promising new opportunities for the advancement of robotics and artificial intelligence.