Why Documentation Matters for AI Context

Documentation as a concept is not new. Documentation as infrastructure — as the primary mechanism by which you give AI agents accurate context — that is the thing people are still figuring out.

I have settled on three documents that live in a docs/ folder at the repo root: config.md, architecture.md, and AGENTS.md. Each serves a different purpose. Together, they give any AI tool, any new team member, and future-me the context needed to work in this codebase without starting from scratch every time.

config.md: The ground truth for how the project is set up

This file answers the questions that are always getting asked. What tech stack? What are the environment variables? What does the local dev workflow look like? How do you run the tests?

The format is deliberately flat and scannable — a dense, current, machine-readable statement of facts about this project. When I change something about the setup, I update this file first. The AI reads it, and the next prompt carries the right context.

The goal is not a full README replacement. It is a living document that stays current because it has to. When it drifts, the AI starts making wrong assumptions.

architecture.md: How the pieces fit together

Where config.md is facts about the project, architecture.mdis the story of how it is structured. The major components, where they live, how they communicate, what the data flow looks like — described with enough precision that an AI generating code for a specific layer knows what system it is operating in.

I use Mermaid diagrams here. They render in GitHub and the AI can read the source syntax directly. No screenshots, no exported images. The diagram is text, which means it is context.

One trap worth knowing, because it bit me: do not leave template placeholders inside a Mermaid block. Mermaid reads { as a node-shape character, so a node like A[Browser / {{Frontend}}] is a parse error and GitHub renders a red “No diagram type detected” box where your architecture should be. Use real labels in the diagram and keep the {{placeholders}}in the prose around it. A broken diagram is worse than no diagram — it reads as a broken doc.

Without this, the AI generates architecturally correct code in isolation that is wrong for your system. With it, suggestions fit. Code slots in. Reviews go faster.

AGENTS.md: Rules of engagement for AI tools

This file is a direct set of instructions for AI agents operating in this codebase: what they are allowed to do, what they should not touch, what conventions matter here. The effect is immediate. The AI stops proposing structural changes you did not ask for. It follows your conventions instead of defaulting to whatever its training data preferred. It behaves like a teammate who read the onboarding docs.

Placement matters more than people expect. A file at the repo root applies to the whole project; one inside a subdirectory cascades on top for work under that path; and a personal global file applies across every project you touch. Project rules go in the repo, machine-specific preferences go in the global file.

Two parts of this file earn their keep. The first is the short list of hard prohibitions — never read the secrets file, never write directly to the database, never rename a persisted identifier without a migration. Each one is a mistake you only want to make once. The second, and the part most people skip, is a set of recipes: a named, repeatable pattern for the tasks you do constantly — new endpoint, new migration, new test — each pointing at the exact existing file to copy. That is what stops the AI inventing a fresh convention every session. And make “write a test for it” an explicit rule, not a hope: an agent will happily ship untested code if you never said otherwise.

The compounding return

The first time you write these three files, it takes a couple of hours. After that, a few minutes to keep them current when something significant changes. The return compounds: every AI-assisted task in this codebase benefits from the context, every new contributor gets onboarded faster, and you stop losing tokens on context restoration at the start of each session.

This is not documentation for documentation's sake. It is context infrastructure. The AI is only as useful as the context you give it. These files are how you give it context systematically rather than ad hoc.