Nvidia’s ‘Cosmos’ AI helps humanoid robots navigate the world

Nvidia announced today that it will release a set of core AI models called Cosmos that can be used for training humanoidsAnd industrial robots Self-driving cars. While language models learn how to generate text through training from copious amounts of books, articles, and social media posts, Cosmos is designed to create images and 3D models of the physical world.

During a keynote presentation at the annual CES conference in Las Vegas, Nvidia CEO said Jensen Huang Show examples of using Cosmos to simulate activities within warehouses. Cosmos was trained on 20 million hours of real footage of “humans walking, moving their hands, manipulating objects,” Jensen said. “It’s not about producing creative content, it’s about teaching AI to understand the physical world.”

Researchers and startups hope these types of foundational models can achieve just that Giving robots used in factories and homes More advanced capabilities. Cosmos software, for example, can create boxes of realistic video footage falling from shelves inside a warehouse, which can be used to train the robot to recognize accidents. Users can also fine-tune models using their own data.

Nvidia says a number of companies are already using Cosmos, including humanoid robotics startups Agility and Figer AI as well as self-driving car companies like Uber, Waabi and Wayve.

Examples of warehouse snapshots created by Cosmos.

Courtesy of Nvidia

Nvidia also announced software designed to help different types of robots learn to perform new tasks more efficiently. The new feature is part of Nvidia’s existing Isaac robot simulation platform that will allow robot manufacturers to take a small number of examples of a desired task, such as grasping a specific object, and generate large amounts of synthetic training data.

Nvidia hopes the Cosmos and Isaac projects will attract companies looking to build and use humanoid robots. Jensen was joined on stage at CES by life-sized images of 14 different humanoid robots developed by companies including Tesla, Boston Dynamics, Agility and Fig.

Along with Cosmos, Nvidia also announced Project Digits, a $3,000 “personal AI supercomputer.” It can run a large language model of up to 200 billion parameters without the need for cloud services from the likes of AWS or Microsoft. It also announced the upcoming next-generation RTX Blackwell GPUs, and incoming software tools to help with the build Artificial intelligence agents.