Nvidia VP of applied research Jonathan Cohen sadi the company has been working on Guardrails' underlying system for "many years" but just about a year ago realized it was a good fit for models along the lines of GPT-4 and ChatGPT.

"We've been developing toward this release of NeMo Guardrails ever since AI model safety tools are critical to deploying models for enterprise use cases," he said.

Guardrails includes code, examples and documentation to "add safety" to AI apps that generate text and speech. Nvidia claims that the toolkit is designed to work with most generative language models, allowing developers to create rules using a few lines of code.

Guardrails can be used to prevent models from veering off topic, responding with inaccurate information or toxic language and making connections to "unsafe" external sources. The idea is that it can keep a customer service assistant from answering questions about the weather or a search engine chatbot from linking to disreputable academic journals.

Cohen said that it will be up to developers to control what is out of bounds for their application with Guardrails.

Nvidia acknowledges that the toolkit isn't imperfect and works better with models that are "sufficiently good at instruction-following such as ChatGPT, and use the popular LangChain framework for building AI-powered apps.

That means that some of the open source options out there will not be much chop. The fact that it's a part of the company's NeMo framework, means it is available through Nvidia's enterprise AI software suite and its NeMo fully managed cloud service.