A chat templated prompt is a completion prompt written in a special format that presents an
interaction with a language model as a simulated conversation, with each contribution labelled by
participant — user:, assistant:, and usually system:. Underneath it is still completion: the
natural continuation of a prompt ending in assistant: is for the model to answer the preceding
user turn. The format exists because the earliest models worked as bare completion engines, which
proved not to be user-friendly.
Usage
The consequence that matters in practice follows from models being stateless: every execution starts from the same blank slate. To maintain the illusion of a continuing conversation, the software talking to the model keeps its own state and replays the entire existing exchange each time the user adds a turn. Since providers charge for input as well as output tokens, this is why a conversation becomes more expensive the longer it runs — the input grows on every turn. Token Caching is the pricing mechanism that partly offsets this, and the reason coding agents are built to avoid modifying earlier conversation content.
The same format is what a tool call is expressed in. A system turn describes an available tool and how to request it; the model ends its turn with a call; the harness extracts that request, executes it, and appends the result as a further turn before asking the model to continue. On this account a coding agent is not doing anything structurally exotic — it is assembling a chat templated prompt, reading what comes back, and doing it again in a loop.
Related Terms
Token Caching, AI Coding Agent, Prompt engineering, Context engineering, AI Agent, Compaction