NVIDIA's Cosmos Policy helps robots predict what happens next

NVIDIA has introduced Cosmos Policy, a novel robot control framework that leverages large pretrained video prediction models to simplify decision-making in robotics. Unlike traditional robot policies that rely on separate perception, planning, and control modules and require extensive task-specific data, Cosmos Policy post-trains a pretrained video world model (Cosmos Predict) on robot demonstration data. This approach integrates robot actions, physical states, and task outcomes into a unified temporal representation, enabling the model to jointly predict the robot’s next actions, future states, and task success within a single architecture. This reduces architectural complexity and the need for large amounts of robot-specific training data. Benchmark tests demonstrate that Cosmos Policy achieves high success rates on multi-step robotic manipulation tasks, often matching or surpassing existing methods while using significantly fewer training demonstrations. A key advantage is its planning capability at inference time, allowing the model to generate and evaluate multiple candidate action sequences and select those with the best predicted outcomes over longer horizons. This strategic planning enables robots to perform complex tasks