The American robotics startup Generalist AI unveiled yesterday, Wednesday, August 19, the latest version of its AI model designed to help robots learn and carry out physical tasks. According to several engineers, this development could mark the entry of robotics into its own “GPT-3 moment.”
- Unlike most industrial robots, programmed to repeat a task or a defined set of tasks, GEN-1.5 can learn a new task in just a few seconds from a single demonstration — what the startup calls “physical prompting.”
- The model shows an average success rate of 59% across a suite of 10 simple manual tasks — folding a sheet of paper, stacking cups, flipping a phone… — but this figure climbs to 83% after a brief retraining using 5 minutes of data, roughly fifty demonstrations.
- After a brief fine-tuning — a light adjustment of the model’s internal weights over a few minutes of data — GEN-1.5 managed to sweep up debris with a brush with virtually no errors (99% success).
Generalist AI is a startup founded in 2024 by former engineers from Google DeepMind and Boston Dynamics, one of the most innovative robotics companies in the world. It is particularly known for its robot Spot, a quadruped now used by military and corporate entities for reconnaissance and inspection missions, such as in the tunnels of the Paris metro since 2021.
- The comparison of GEN-1.5 with GPT-3, according to Generalist AI, rests on the model’s ability to rapidly learn new tasks from a single or a few examples — one of the defining features that marked the advent of OpenAI’s language model.
- However, for Yu Xiang, a robotics researcher at the University of Texas at Dallas, the comparison with GPT-3 remains premature: language provides a unified representation that everyone uses to interact with the model, whereas robots exhibit a wide variety of architectures, grippers, and hands.
- The real test, in his view, will be whether GEN-1.5 can transfer to very different robots while preserving its capabilities.
One of the striking features of GEN-1.5 is that the model’s pretraining contains no simulation data. It was conducted exclusively from real physical interaction data captured in everyday environments — homes, warehouses, factories — via the company’s “data engine.”
For GEN-1, its previous model unveiled in April, Generalist AI indicated it relied on roughly 500,000 hours of real-world data.
- The startup acknowledges that it is “difficult to determine precisely why these abilities emerge” from the pretraining of GEN-1.5. It hypothesizes that the model may have learned to detect and reproduce repetitive patterns inherent to physical work, much as language models chase motifs in generic sequences.
The ability for robots to learn and perform new tasks after only a few minutes of training could yield substantial productivity gains. China, which alone accounts for 43% of the global fleet of industrial robots, would be one of the best-positioned countries to benefit.
- Beijing installs more industrial robots than the rest of the world combined, according to the latest report from the International Federation of Robotics (IFR), published in September.
- In 2024, more than half of the robots installed in Chinese factories were produced domestically.