This paper, by four authors at EPAM Systems, asks whether the data available in an application’s repository is sufficient for an LLM to deploy a working multi-service environment, and what it cannot convey. It observes that prior work on automated environment creation mostly targets a single test-oriented container, and that LLM-based Infrastructure as Code generation usually starts from an expert-written description of the desired infrastructure; here the model instead has to recover the deployment from source code, manifests and configuration, with the repository’s own Dockerfiles and Compose files withheld.
The experiment used GPT Codex 5.3 through GitHub Copilot on an Apple M1 Pro (ARM64) host and three
public repositories from Docker’s sample collections: example-voting-app (Python, Node.js and
.NET services with Redis and PostgreSQL), react-rust-postgres (React, Rust/Actix Web and
PostgreSQL) and react-java-mysql (React, Java Spring Boot and a MariaDB/MySQL database). Each
generated environment was judged by a deterministic end-to-end HTTP request through the whole service
pipeline — rather than by syntax, image builds or textual similarity — and then compared manually with
the developer-authored artifacts.
Key Points
- All three generated environments became functionally operational: two on the first build and the Rust one after its base image was changed from Rust 1.85 to 1.88 to satisfy the current Actix Web dependency, giving a first-build success rate of 2/3 and a final rate of 3/3.
- The model correctly reconstructed language ecosystems, application ports, infrastructure services,
hostnames and
depends_onrelationships in all three cases, identified a .NET background worker with no HTTP endpoint, discovered a frontend proxy configured insetupProxy.js, and inferred a Docker secrets mechanism from a baredb/password.txtfile. - Where credentials existed only in the withheld Compose file, the model invented alternative values but propagated them consistently, and the environment still passed; the authors conclude that for underdetermined values, internal consistency may matter more than reproducing the developer’s literal values.
- It consistently omitted deployment intent: network segmentation (0/3, a flat network every time), multi-stage Dockerfiles, dependency-layer caching and live-reload mounts (0/3 each), production frontend serving through Nginx (0/2), restrictive backend-port exposure (0/2, backend ports were published to the host) and cross-platform build logic (0/3). The authors call this gap between required deployment knowledge and what a repository reveals the signal gap.
- The authors classify DevOps knowledge as directly observable, derivable, underdetermined or intent-only, and argue that observable and derivable knowledge was generally reconstructable while underdetermined and intent-only knowledge needs explicit specification or controlled defaults.
- On that basis they propose Specification-Driven DevOps, in which repository evidence supplies what is observable and a minimal explicit deployment specification supplies the rest, and sketch its fields, conflict-resolution principles, validation rules and a traceability model. This specification is analytically derived and was not tested by regeneration.
Notes
The paper situates itself between LLM-based Dockerfile and environment synthesis work, IaC generation benchmarks, and specification-driven approaches such as code-text-code re-engineering and architecture metamodels, several of which are earlier work by the same authors. The authors present it as an exploratory feasibility experiment and list its limits: three relatively simple, publicly available repositories the model may have encountered in training; a single model and a single generation attempt per repository with an incompletely documented interaction record; one HTTP oracle per repository with no security, performance or failure testing; manual structural comparison by the authors without a formal rubric; a manual rather than LLM-driven fix for the Rust version; and no experimental validation of the proposed specification.