NVIDIA enters this wiki mainly through its AI security work, and also as the publisher of open-source agent tooling. The team responsible is the NVIDIA AI Red Team, which tests AI and agentic systems adversarially and publishes what it finds on the NVIDIA Technical Blog.
Activities & Products
The Red Team’s practice, as visible in the work summarized here, is to construct a working attack against a real deployed system, disclose it to the affected vendor, and publish the finding together with the disclosure timeline and the vendor’s response — including where the vendor declined to make changes. Its report of an Indirect AGENTS.md Injection attack against OpenAI Codex followed that pattern, running from the initial report through the vendor’s conclusion that the attack did not materially raise risk beyond compromised-dependency scenarios.
An earlier disclosure followed the same shape against LangChain: the Red Team identified and verified three vulnerabilities in that library’s chains, all reachable through Prompt injection, and published them once the affected components had been removed from the core library and with the maintainers’ approval. That write-up records NVIDIA requesting a CVE itself after what it describes as a lack of immediate mitigation, and sets out the reasoning behind publishing — that the issues were severe but confined to specific chains, and that the technique was by then widely understood. It also generalizes beyond the finding, arguing that all model output should be treated as potentially malicious, which is the position developed under Control-Data Plane Confusion.
Alongside the research, NVIDIA publishes tooling that its own security recommendations point to: garak, described there as an LLM vulnerability scanner for evaluating models against known prompt injection weaknesses, and NeMo Guardrails, for filtering and protecting LLM inputs and outputs. It also publishes training material in this area, including a self-paced course on adversarial machine learning, and has run AI red team training at Black Hat. Its NeMo offering is also named in that earlier write-up as a service for supporting LLM applications and integrations.
Beyond security, NVIDIA publishes NVIDIA NeMo Agent Toolkit, an open-source library under the Apache 2.0 license that works alongside existing agent frameworks to add profiling, observability, evaluation and optimization to AI agents. Its README ties the toolkit to NVIDIA’s own infrastructure through NVIDIA NIM model endpoints, used in its introductory example, and an experimental integration with NVIDIA Dynamo for improving agent performance at scale.