Nvidia's New AI Safety Team: Open Models & Security Focus

Nvidia is quietly building a new AI safety and security engineering team. The company posted several job listings late last month to staff this new group. These new hires will evaluate AI agents before they are deployed. They will also build AI-powered tools to fix software vulnerabilities. One job listing states the team believes open-weight models and transparency are key for American AI leadership and cybersecurity defense. This move suggests Nvidia is making AI safety a larger priority. This comes as the company focuses on a future driven by open-weight AI models. These models make their trained "weights" publicly available. Nvidia CEO Jensen Huang recently urged US policymakers to support open models, stating they improve safety and cybersecurity. Days later, Nvidia joined the Open Secure AI Alliance, a group creating open-source security tools for AI. This alliance includes 120 companies. This push towards open models and AI security also aligns with Nvidia's business goals. Open models increase the number of potential customers, which drives demand for Nvidia's AI chips. As AI agents become more sophisticated, trust in their safety will be crucial for widespread adoption.