Google rolls out new Gemini model that can run on robots locally

Google DeepMind has introduced Gemini Robotics On-Device, a new language model designed to run locally on robots without needing an internet connection. This model builds on the previous Gemini Robotics version by enabling direct control of robot movements through natural language prompts, allowing developers to fine-tune it for various applications. According to Google, Gemini Robotics On-Device performs nearly as well as its cloud-based counterpart and surpasses other unnamed on-device models in general benchmarks. In demonstrations, robots equipped with this local model successfully performed tasks such as unzipping bags and folding clothes. Although initially trained for specific tasks, the model was later adapted to work on different robot platforms, including the bi-arm Franka FR3, which managed to handle new scenarios and objects it had not encountered before. Additionally, Google DeepMind is releasing tools that allow developers to train robots on new tasks by providing 50 to 100 demonstrations using the MuJoCo physics simulator. This development aligns with broader industry trends, as companies like Nvidia, Hug