This paper takes spec-driven development as a promising response to the brittleness of multi-step agentic coding, and then argues that structuring the workflow is not by itself enough. Its authors name the residual failure mode Context Blindness: an agent’s intermediate artifacts can be internally coherent while being incompatible with the repository as it actually exists, with symptoms including references to non-existent APIs, proposed file paths that do not exist, and violations of local architectural or stylistic conventions. Because these errors surface late, during implementation or test execution, the agent backtracks and revises earlier artifacts, compounding the problem across stages.
The system proposed in response, Spec Kit Agents, augments the staged Specify → Plan → Tasks → Implement workflow with what the authors call a context-grounding layer, deliberately placed outside the core agent prompts so that traces stay auditable and tool access can be granted selectively. Discovery hooks run before each phase as a read-only prober, gathering evidence about the codebase with repository inspection tools to surface project-specific conventions, existing APIs and relevant modules. Validation hooks run after each phase: on earlier artifacts they check structural and referential constraints such as whether file paths referenced in the plan exist and whether required libraries are present, and after implementation they execute repository checks such as unit tests and linters. The pipeline is orchestrated as a state machine over two roles — a product manager agent responsible for clarifying requirements and prioritization, restricted to repository analysis and version-control inspection, and a developer agent permitted to edit files and run repository commands.
The evaluation covers 128 runs over 32 feature tasks in five open-source repositories, with each task run under four configurations: Baseline, which proceeds directly to implementation; Augmented, which adds the hooks to that direct flow; Full, which executes the full staged workflow; and Full-Augmented, which adds the hooks to Full. Generation is separated from evaluation: the agents run through the Claude Code CLI routed to an Anthropic-compatible endpoint backed by MiniMax-M2.5, while Claude Opus 4.6 acts as an independent judge scoring outputs on a 1–5 scale across completeness, correctness, style and maintainability. A small blinded human review is conducted on a subset using the same rubric.
Key Points
- Names context blindness as a failure mode that survives the adoption of a staged spec-driven workflow, and argues it should be addressed by making grounding and validation explicit workflow operations rather than in-trajectory behaviors of the generating agent, which the authors hold makes grounding sensitive to prompt design and context-window noise.
- Reports that Full-Augmented achieves the strongest quality within the longer-budget workflow family, improving the composite judged score from 3.51 to 3.66, a difference the authors report as statistically significant on the paired subset of completed tasks under a Wilcoxon signed-rank test at p < 0.05.
- Reports that repository-level test compatibility remains at 99.7–100% across configurations, which the authors read as the quality gains not coming at the expense of breaking existing project behavior.
- Reports an ablation in which both partial variants improve over Full — validation-only reaching 3.57 and discovery-only 3.53 — with the combined design strongest at 3.66, which the authors take as evidence that the two components are complementary.
- Reports 58.2% Pass@1 on SWE-bench Lite with the hooks enabled against 56.5% for the same framework without them, and states that the orchestration framework is model-agnostic even though all experiments used MiniMax-M2.5 as the base model.
- Characterizes the benefit as earlier detection and prevention of compounding context errors rather than a dramatic jump in average score, and reports the hooks add modest overhead in the shorter-budget family but a larger cost in the longer one, making the approach most appropriate for higher-risk or higher-complexity tasks.
Notes
The authors compare their contribution to prior work on two axes. Against tool-augmented grounding, they position discovery hooks as phase-scoped evidence collection rather than best-effort retrieval, arguing this makes grounding more repeatable and inspectable and less coupled to the main agent’s generation. Against verification work, they note their contribution differs in both when and what is validated: intermediate artifacts are checked before code generation rather than concentrating verification after implementation, while post-implementation executable checks are retained as a final gate.
The paper reports that the gain from augmentation is not uniform across SWE-bench Lite repository families. The authors observe stronger improvements where failures are tightly coupled to unit-tested, test-adjacent code paths, and less reliable gains on repositories where failures originate in deeper application or library logic that local unit tests expose only weakly — noting that where available tests under-specify an edge condition, the hooks may anchor on incomplete signals. They summarize this as augmentation helping most when tests directly exercise the underlying defect.
The paper carries no threats-to-validity section; some limits of the setup are stated in passing. Human-facing plan-review checkpoints were auto-approved, runs exceeding their time budget were terminated and marked as failures, and latency is compared only within each budget family rather than across them.