Nvidia GTC Taipei: New AI Models for Robots & Cars

Nvidia is making a significant push into physical AI, unveiling new models for robots, autonomous vehicles, and video systems at GTC Taipei. The company launched Cosmos 3, a new "omnimodel" that processes various data types like text, images, and audio. It helps developers generate synthetic training data and predict future world states, avoiding the need to recreate real-world situations. Cosmos 3 can analyze video for smart cities, generate photorealistic video sequences for rare events, and produce motion data for robots learning tasks. Nvidia also introduced Alpamayo 2 Super, a scaled-up driving model for Level 4 autonomous vehicles. This model has 32 billion parameters, a significant increase from previous versions, aiming to improve spatial understanding and handling of rare situations. It also outputs "meta-actions" like "lane change" to a downstream planner. These advancements could lead to more capable and efficient AI systems in robotics and autonomous driving.