NVIDIA’s GTC Taipei keynote on June 1 didn’t just drop new models—it lit the fuse on the physical AI revolution. Jensen Huang took the stage in Taipei amid the COMPUTEX buzz and declared the dawn of “agentic AI factories”: always-on systems that reason, simulate, and act in the real world. At the heart of it all? Cosmos 3, the first fully open omnimodel for physical AI, paired with the scaled-up Alpamayo 2 Super for robotaxis and a wave of infrastructure announcements that position NVIDIA as the full-stack architect of the intelligence era.[1]
If you’ve been tracking the shift from chatbots to systems that can actually do things in warehouses, on roads, or on factory floors, this keynote was the clearest signal yet. The buzz on X and in tech media today is electric—and for good reason. These releases aren’t incremental; they’re foundational.
Cosmos 3: The Open Frontier Omnimodel for Physical AI
Cosmos 3 is NVIDIA’s boldest open release yet: a world foundation model (WFM) built on a breakthrough mixture-of-transformers (MoT) architecture. It unifies vision reasoning, world generation, and action prediction in one system—natively handling text, images, video, ambient sound, and actions with leading physics accuracy.[2]
Previous physical AI efforts often required stitching together separate models for perception, simulation, and control. Cosmos 3 collapses that stack. Its reasoning transformer first interprets scenes (object interactions, motion, spatial-temporal relationships), then hands off to an expert generation transformer for physically grounded outputs like synthetic video or robot action trajectories.
Trained on one of the largest multimodal physical AI datasets—billions of samples across modalities—it dramatically cuts training and evaluation cycles from months to days. On benchmarks, Cosmos 3 tops leaderboards for open models in vision reasoning, world simulation, and action generation.[3]
The lineup includes:
- Cosmos 3 Super: Highest physics accuracy and generation quality for post-training robotics and AV models.
- Cosmos 3 Nano: High-quality outputs in fractions of a second, runnable on workstation-grade hardware like the RTX PRO 6000.
- Cosmos 3 Edge (coming soon): Optimized for real-time edge inference.
NVIDIA is open-sourcing models, training scripts, deployment tools, and datasets via Hugging Face and GitHub. Developers can customize with post-training, generate synthetic data, and deploy as NIM microservices. The Cosmos Coalition—with partners like Agile Robots, Black Forest Labs, Runway, and Skild AI—will push open world models forward collaboratively.[4]
Early adopters span robotics (Doosan, LG, Samsung), AVs (Li Auto), and industrial AI. As Huang put it, this is the “big bang of physical AI.”
See our guide on building physical AI systems with world models
Alpamayo 2 Super: Scaling Reasoning for Level 4 Robotaxis
Autonomous driving just got a massive reasoning upgrade. Alpamayo 2 Super is NVIDIA’s most powerful open reasoning VLA (vision-language-action) model yet—a 32-billion-parameter giant that reasons, plans, and acts across the full driving stack.[5]
Scaling from the prior 10B-parameter Alpamayo generations, it moves beyond simple trajectory generation to human-like perception and decision-making. Key upgrades include:
- Full-surround perception (360° from front/side/rear cameras).
- Meta-Actions (high-level decisions like “yield” or “lane change” alongside trajectories and chain-of-causation traces).
- Reasoning auto-labeling with 2D grounding—compressing annotation from months to days.
- Improved handling of long-tail, rare scenarios where traditional imitation learning fails.
Designed as a “teacher model,” Alpamayo 2 Super can be distilled into smaller onboard models running on DRIVE AGX Thor. It ships with supporting tools: AlpaGym (closed-loop RL framework in simulation for learning from consequences), OmniDreams (photorealistic generative world model for rare scenario generation), and new physical AI agent skills powered by Omniverse NuRec for neural reconstruction of real-world fleet data.[5]
Alpamayo has already seen ~400,000 downloads since launch and won COMPUTEX recognition in Vehicle Technology. Partners like Uber, Lucid, and Jaguar Land Rover are early explorers. This isn’t just for robotaxis—it’s a blueprint for any embodied reasoning system.
Agentic AI Factories: From Token Production to Autonomous Operations
The bigger picture from the keynote: AI is no longer just about models—it’s about AI factories, massive, always-on infrastructure optimized for inference, reasoning, and agentic workflows at scale.[3]
Vera Rubin (NVIDIA’s next-gen platform with custom Vera CPU) is now in full production, powering these factories with rack-scale systems (MGX architecture) that deliver dramatically higher agent throughput—up to 10x in some comparisons—while improving energy efficiency. The supply chain in Taiwan alone involves over 500 partners and millions of square feet of factory space.[6]
Complementing this is the NVIDIA Factory Operations Blueprint (FOX): a reference design for an autonomous “factory manager agent.” It continuously monitors real-time data (machines, quality systems, alerts), reasons across it, and orchestrates fleets of specialized agents using NemoClaw and AI skills. Early results from partners like Foxconn show 80% faster root-cause analysis, 15% labor productivity gains, and 10% fewer machine failures.[7]
These “always-on” factories turn compute directly into revenue-generating intelligence. Huang emphasized that tokens are now profitable units, and useful AI is a net job creator. The infrastructure (Vera Rubin + MGX + 800VDC power + Spectrum-X Photonics) is built for exactly this: resilient, scalable, agent-driven operations.
See our guide on scaling AI factories with MGX and Vera Rubin
Humanoids and the Full Physical AI Stack
NVIDIA didn’t stop at models and factories. The keynote also unveiled the NVIDIA Isaac GR00T Reference Humanoid Robot—an open reference design combining a Unitree H2 Plus chassis, Sharpa five-fingered tactile hands (75+ degrees of freedom total), Jetson Thor compute, and the Isaac GR00T platform.[8]
This gives researchers and developers a ready-to-use, simulation-to-real pipeline for generalist humanoid development. Paired with Cosmos 3 for world modeling and GR00T models for behavior, it accelerates the path from data generation to deployment.
The full stack—Cosmos for world understanding, Alpamayo for specialized domains like driving, FOX for operations, GR00T for embodiment, and Vera Rubin for the underlying compute—creates an end-to-end ecosystem for physical AI.
Why This Matters Now: The Shift to Embodied, Agentic Intelligence
These announcements arrive at a pivotal moment. Generative AI proved the power of scale; now the industry is moving to systems that interact with the physical world. Cosmos 3 and its companions solve the “hardest data problem” in robotics—perspective-specific, physics-grounded training data—while open-sourcing lowers barriers dramatically.
For enterprises, AI factories promise autonomous operations that react in real time rather than after the fact. For developers, the open models and toolkits mean faster iteration on robots, AVs, and smart spaces. For the broader ecosystem, the Cosmos Coalition and Taiwan’s manufacturing muscle signal a collaborative, global push.
Expect rapid adoption in manufacturing, logistics, automotive, and beyond. The “always-on” nature of these factories—running continuous inference and agent loops—will redefine what’s possible at scale.
FAQ
What is Cosmos 3 and how does it differ from previous Cosmos models?
Cosmos 3 is NVIDIA’s latest open world foundation model for physical AI, featuring a mixture-of-transformers architecture that unifies reasoning, multimodal generation (text, image, video, sound, action), and physics-accurate simulation in one system. Unlike earlier versions focused more narrowly on generation, Cosmos 3 excels at native vision reasoning and action prediction, topping open-model benchmarks while being fully open-sourced with extensive toolkits.[2]
How does Alpamayo 2 Super improve on earlier Alpamayo versions for autonomous driving?
Alpamayo 2 Super scales to 32B parameters (from 10B), adds full-surround perception, meta-actions, reasoning auto-labeling, and stronger chain-of-causation traces. It functions as a teacher model for distillation and comes with closed-loop training tools like AlpaGym and OmniDreams for safer, more robust Level 4 robotaxi development.[9]
What are agentic AI factories and why are they central to NVIDIA’s vision?
Agentic AI factories are large-scale, always-on infrastructure (powered by platforms like Vera Rubin) optimized for training, inference, and running multi-step reasoning agents at production scale. They shift AI from reactive tools to proactive systems that observe, plan, and act—enabling everything from autonomous factory management (via FOX blueprint) to physical AI deployment.[10]
Is Cosmos 3 available now, and where can developers get started?
Yes—Cosmos 3 Super and Nano are available immediately via Hugging Face (models), GitHub (tools/scripts), and NVIDIA’s build platform. Cosmos 3 Edge is coming soon. The Cosmos Coalition and partner ecosystems further support customization and deployment.[11]
What excites you most about these physical AI breakthroughs—building your first Cosmos-powered robot, scaling an agentic factory, or something else entirely? Drop your thoughts in the comments below.