NVIDIA Expands Agent Toolkit with Omniverse 3D Libraries
NVIDIA Expands Agent Toolkit with Omniverse Libraries to Automate 3D Simulation Workflows
NVIDIA has updated its Agent Toolkit with new open-source Omniverse libraries, enabling autonomous AI agents to validate assets, simulate physics, and prepare 3D scenes inside existing design applications like Blender and Houdini.
Nvidia has announced an expansion of its Agent Toolkit via the integration of Omniverse libraries. The collection of software components provides AI agents with specialised tools and skills to embed physical AI capabilities directly into existing 3D software applications and creative workflows.
Automating scene inspection, issue flagging and asset preparation, the libraries aim to streamline the complex process of converting standard 3D design content into simulation-ready environments for robotics, industrial manufacturing and autonomous systems.
Speaking on the launch, Jensen Huang, founder and chief executive of Nvidia, said: “The physical AI era will be built in simulation first. Nvidia Agent Toolkit with Omniverse libraries brings AI agents into the 3D tools developers already use, helping build the simulation-ready worlds where robots, factories and autonomous systems are trained and tested long before they reach the real world.”
Sensor simulation, physics and OpenUSD standards
Now openly available on GitHub, the new Omniverse libraries feature three primary skill sets designed to enhance spatial and physical modelling in 3D environments. The ovrtx library provides sensor simulation, enabling applications to generate realistic camera, lidar and radar outputs from 3D scenes, allowing developers and AI agents to test how physical AI systems perceive virtual environments. For physical behaviour, the ovphysx library utilises GPU-accelerated physics to replicate real-world properties such as collisions, mass, friction and motion within 3D scenes.
Additionally, CAD-to-SimReady skills allow technical artists and designers to convert computer-aided design (CAD) data into standardised OpenUSD (Universal Scene Description) assets containing the essential physical properties required for virtual testing.
To demonstrate the practical integration of these libraries, Nvidia introduced a blueprint for implementing Omniverse capabilities into Blender, showcased through a sample workflow named ‘SimReady’ Blender. The integration demonstrates how software makers can embed sensor simulation and physics validation into open-source 3D tools while ensuring creators retain complete creative control over the final output.
Software integration across 3D and CAD platforms
Several major design software developers and startups have begun integrating the Omniverse libraries into their production environments, with blueprints available on GitHub.
Kim Davidson, president and chief executive of SideFX, said: “Procedural 3D creation is essential to building the complex, controllable worlds needed for simulation, robotics and industrial AI. With Nvidia Omniverse libraries and OpenUSD, SideFX is exploring how agent-ready tools can support Houdini workflows, helping technical artists review, test and prepare procedural content for simulation while staying in control of the creative process.”
Similarly, PTC’s Onshape CAD and product data management platform is adopting OpenUSD and ovrtx to connect cloud-native design pipelines with physical simulation. Neil Barua, president and chief executive of PTC, said: “Engineering teams are seeking more connected ways to design, collaborate and simulate throughout the development process. PTC’s work with Nvidia supports that broader vision, while Nvidia Omniverse libraries help enable simulation-ready workflows that bring validation and testing closer to where products are designed.”
Startups including Palatial, Lightwheel, ForgeCAD and Moonlake AI are similarly using the skills to automate asset generation and scene validation from CAD files and text prompts.
The agentic workflows support on-device execution across hardware setups ranging from compact Nvidia RTX Spark systems to Nvidia DGX Station workstations. This local infrastructure aligns with the practical requirements of running local AI in professional 3D pipelines, ensuring that creative studios and engineering teams can automate asset preparation without sending sensitive project files to external servers.
RTX Spark hardware models will be released this autumn from manufacturers including Asus, Dell Technologies, HP, Lenovo, Microsoft Surface and MSI, followed by Acer and Gigabyte.