The robot arm in the simulation reaches for a box, knocks it slightly off balance, recovers and tries again. Nothing in the video is real, but everything about it is physical: gravity, friction, momentum, cause and effect. This is the world Nvidia is trying to sell, and on May 31, at the GTC Taipei keynote during Computex, chief executive Jensen Huang rolled out the latest version of the software that generates it. Cosmos 3 is a family of what Nvidia calls omnimodal world models, systems that jointly understand and generate language, images, video, audio and action, built as a general-purpose backbone for physical AI.
The pitch is straightforward. Training a robot in the real world is slow, expensive and occasionally dangerous. A robot that can imagine what happens next, that can watch a scene and predict how it will unfold, can practice millions of times in simulation before it touches a physical object. The same logic applies to self-driving cars, which can rehearse rare and dangerous traffic situations that would be reckless to create on real roads. Cosmos 3 is designed to be the infrastructure layer for that kind of training, unifying what previously required a chain of specialist models into a single architecture.
The architecture is the interesting part. Cosmos 3 uses a mixture-of-transformers design with a two-tower structure that separates reasoning from generation, and it differs from earlier generative models in one critical way: it can emit action. Beyond rendering a plausible next frame, the model can output numerical control signals, joint angles, gripper positions and trajectory points, that a robot controller or an autonomous vehicle stack can execute directly. That is the difference between a model that imagines the world and a model that can be wired into a control loop. Nvidia calls the approach a unification of physical understanding, world generation and action generation in a single forward pass.
The release includes real deliverables. Nvidia has published the Super variant, a 64-billion-parameter model, and the Nano variant at 16 billion parameters, with an Edge model at 4 billion parameters to follow, along with six synthetic datasets, training scripts and an evaluation benchmark, all under the OpenMDW-1.1 license managed by the Linux Foundation. Weights are available on Hugging Face, and deployment is offered through Nvidia’s NIM microservices. The company also released a DROID policy model for robot manipulation, a demonstration of how Cosmos 3’s world knowledge can be adapted into concrete control policies.
The benchmarks, all vendor-run, claim top positions among open models on a stack of physical-AI evaluations, including Physics-IQ, PAI-Bench, RoboLab, RoboArena and a traffic-anomaly reasoning leaderboard. Nvidia also says Cosmos 3 leads open leaderboards for text-to-image and image-to-video generation. Skeptics will note that Nvidia’s competitors, including Google’s DeepMind and a crop of startups building world models, are racing for the same territory, and that vendor-reported scores tend to look best on vendor-selected tests. But the substance of the release, open weights plus datasets plus recipes, gives researchers something concrete to evaluate, which is more than most competitors have shipped.
The strategic logic for Nvidia is clear. The company’s revenue depends on selling GPUs, and the marginal use of those GPUs is increasingly AI training and inference. Physical AI, robots and autonomous vehicles, is the next large source of demand, one that extends the AI buildout beyond chatbots and into factories, warehouses and roads. A world model that makes robots easier to train makes GPUs easier to sell, and an open model that lowers the barrier to entry for robotics research converts more of the industry onto Nvidia hardware and software. The pattern echoes Nvidia’s history with CUDA, the programming platform that turned its graphics chips into the default compute for AI by making them the easiest option for developers.
The release also reflects a bet on where AI is heading. Language models predict the next token; world models predict the next state of the world. If embodied AI, machines that perceive, reason and act, becomes the industry’s next act, then the companies that own the simulation layer will own a chokepoint comparable to the one Nvidia already holds in training compute. Huang has been talking about physical AI for years, and Cosmos 3 is the clearest expression yet of that vision: not a single product but a platform, built on the assumption that the future of intelligence includes bodies.
None of this is guaranteed. The gap between a world model that generates convincing video and one that generates physically accurate predictions for a real robot remains wide, and critics note that generative models are prone to inventing plausible but wrong futures. Nvidia acknowledges the challenge, positioning Cosmos 3 as a starting point for mid-training rather than a finished solution, a base that downstream developers specialize for their own tasks and embodiments.
For now, the release puts Nvidia at the center of the physical AI conversation, with open weights, a clear architecture and a distribution channel that spans cloud, workstation and edge. Whether Cosmos 3 becomes the default foundation for robot training will depend on how well its simulated physics hold up in real deployments. Nvidia is betting that the answer is good enough, and that the companies training the world’s robots will pay for the compute either way.
