The IBM Watsonx Orchestrate Agent Development Kit (ADK) is a set of Command Line Interface (CLI) utilities and a Python-based library that gives tool builders a tooling environment to configure, deploy, and manage agents and tools within Watsonx Orchestrate. A Framework for Testing and Adapting REST APIs as LLM Tools describes a REST-API testing framework deployed as part of the ADK’s alpha release and specifically designed for the tool-builder persona.
Capabilities
As part of the ADK, the framework in A Framework for Testing and Adapting REST APIs as LLM Tools lets a tool builder test a tool with an agent. Currently, that support is provided through a CLI, where a builder can generate test cases for a tool and further evaluate the tool with an agent; the builder then receives a categorized error report — see A Framework for Testing and Adapting REST APIs as LLM Tools’s error taxonomy — along with template-based recommendations for improving the tool’s definition.
Adoption & Ecosystem
As of the paper’s writing, tool builders using the ADK’s testing feature had wrapped more than 600 APIs as tools spanning a range of domains, including IT, human resources, finance, procurement, and sales. Before this testing feature was introduced, builders manually created tools and tested only 5-6 test cases per tool; the paper reports the automated test generation and recommendations reduced build effort per tool by about 30% (roughly two days per tool), for a reported total of over 1,200 person-days saved to date. The generated test cases are also reused for continuous and regression testing to verify updated tools continue functioning correctly.