The Berkeley Function Calling Leaderboard (BFCL), introduced by Yan et al. (2024), is a benchmark dataset for evaluating how well large language models perform function-calling (tool-calling) tasks. A Framework for Testing and Adapting REST APIs as LLM Tools uses BFCL’s V3 release as part of the training data for fine-tuning its own natural-language test-case generation model.
Contents
Each BFCL example pairs a natural-language user query with a desired function specification (comparable in structure to an API specification) as input, and a structured JSON object naming the correct function, its parameters, and their values as the expected output.
Provenance
BFCL was introduced by Yan et al. (2024) as a benchmark for function-calling in large language models. A Framework for Testing and Adapting REST APIs as LLM Tools draws specifically on its V3 release.
Use
A Framework for Testing and Adapting REST APIs as LLM Tools combines 400 of BFCL’s simpler API examples (each with a maximum of 10 parameters) with 120 additional human-curated, domain-specific API examples of its own, and uses 70% of that combined pool — with 15% each held out for validation and test — to fine-tune a Granite-3B-Code-Base model (via LoRA) to generate natural-language test-case utterances.