All posts

Release Awareness: Why Must an AI Agent Know What Just Changed?

An agent that missed your last release answers about a product that no longer exists. What release awareness is, how to test yours in three questions, and what to connect to get it.

Because an agent that missed your last release answers about a product that no longer exists. Release awareness is an agent's ability to know what changed in the product since a reference point, and to account for it before asserting anything. Without it, every deploy opens a window during which all frozen knowledge is wrong, with the same confidence it had the day before.

The definition, in one sentence

Release awareness: an AI agent's ability to know what changed in the product since a reference point — the last release, the start of the month, the version the asking customer runs — and to answer accordingly.

It is not a property of the model. A larger model does not become better informed about your Tuesday deploy: it has no way of hearing about it. It is a property of the sources the agent consults at answer time. An agent is release aware if, and only if, one of its sources contains the changes and can date them.

Which leads to a result that surprises teams: release awareness is not a prompting problem. You can instruct an agent to "check whether anything changed recently". With nothing to consult, it will comply anyway. It will produce a plausible answer about the previous state.

The post-release window, where the cost lands

The deploy ships. In the hours that follow, two curves cross: question volume rises, and the quality of available knowledge falls.

It falls everywhere at once, which is what makes this expensive. The documentation was written before. The semantic index was built before. Support macros answer for the previous behavior. The model's training data comes from an even older "before". None of these sources flags its own lag.

Take a common case, as an illustration: a release adds a signature to outgoing webhooks. The change is small, two lines of diff. For the next three days, an integrator whose verification now fails asks the support agent about it. The agent reads the "Webhooks" page, which faithfully describes the previous format, and confirms to the customer that nothing changed. The ticket escalates, a developer gets interrupted, and the answer was sitting in a dated commit.

The window closes when someone catches the docs up by hand. That human latency, repeated at every release, is the real duration during which your agents are wrong. It is also the sharpest form of documentation drift: measured in hours here, not quarters.

The three-question test

You do not need an architecture audit to find out where you stand. Ask these three questions of the agent your teams or your customers already use, today, and read the shape of the answers rather than their tone.

Question 1 — "What changed in the product since the start of the month?" Without release awareness, the answer is a product summary: features described in the present tense, not a single date. With it, the answer is a short dated list where every line points to an identifiable change.

Question 2 — "Why did this behavior change?" Without it, the agent explains the current behavior and stops: it never had access to the transition, only to the state. With it, it walks back to the diff that introduced the change and to its intent. This is the question that separates the two regimes most sharply, and we covered it in the article on answers pulled from git history.

Question 3 — "Does the last release break anything for a customer calling the API?" Without it, you get reassuring generalities about versioning policy. With it, you get the affected endpoints, or an explicit answer that none are. Your integrators ask this one first, and it deserves its own treatment.

The signal to watch is not the error. It is the confidence. An agent without release awareness never says "I don't know what changed": it has no way of knowing that it doesn't. It answers about the past, in the present tense.

Ready to see what your team shipped?

What you need to connect to get it

Three conditions, and they are not interchangeable.

A source that contains transitions. An index describes states; changes live between two states. Git history is the one source you already own that keeps both versions of every line. That is the underlying difference between indexing and consulting, which we set out in MCP vs RAG.

Freshness tied to your deploys, not to a calendar. A nightly reindex leaves a one-day window open, every day. The right clock is the one that ticks on pushes.

A channel to query during the conversation. The agent has to fetch when the question arrives, rather than receive a context prepared in advance. That is the job of an MCP server.

This is the regime DeployIt works in. DeployIt reads the real code (resynchronized on every push), the git history (commits + diffs) and customer data through its own MCP servers, and verifies in the code before asserting. Every project gets a product expert your agents can query: Claude, Cursor, a support agent such as Intercom Fin, or your own.

"We'll just reindex on every deploy"

That is the first instinct, and it is a good one. A deploy hook that triggers reindexing moves your sources closer to the present, and it is worth doing whatever comes next.

Two blind spots do not close with frequency, though. First: reindexing more often brings the index closer to the present without letting the past in. Questions 1 and 2 of the test stay unsourced, because they are about transitions and an index keeps no trace of those. Second: if you build on commit messages, your answer quality inherits theirs. DeployIt's answers are derived from the diffs themselves, which holds up even when the history is made of "wip" and "fix".

Then there is the case frequency never reaches: the people asking who have no repository credentials. An agent platform, an agency client under NDA, an external integrator — none of them will get access to your GitHub, and no reindexing cadence changes that. They need a channel for answers, not access to code.

"So this tool reads our whole repository"

The objection is fair, and it always arrives at the right moment. To date a change, something has to read what changed.

The scope is narrow and verifiable: read-only GitHub connection, one server per project, revocable at any time. Answers are sourced from commits and diffs, with no raw source returned: the integrator in our example learns that webhooks have been signed since Tuesday, without reading your sending module. The code never leaves the repository.

Mini-FAQ

Does release awareness only matter for customer-facing agents? It shows up there first, because a wrong answer costs a ticket. Your internal product agent has the same blind spot, though: it answers your product manager about a product that is a few days old. Same defect, one witness fewer.

Should we clean up our documentation first? Nothing requires it, and the reverse order works better. Knowledge derived from code hands the writer a dated list of what to fix — docs benefit from the mechanism instead of preceding it.

How far back should an agent be able to look? Two windows cover most needs: since the last release, for support and integrators, and since the version the customer runs, for accounts that upgrade late. Beyond that, demand thins out.

What about the public libraries our code depends on? Others cover that ground: Context7 and DeepWiki handle libraries and public repositories, Mintlify generates an MCP from written docs. DeployIt covers your private product, derived from the code itself, fresh on every deploy.

An agent is only as fresh as its oldest source

You can swap models, extend the context window, refine the prompts: none of that teaches an agent what shipped on Tuesday. Only a source that contains the changes, and can date them, will.

Ask the three questions. The shape of the answers tells you, in five minutes, which regime your agents are working in.

DeployIt's MCP server is in early access / design partner program: if your agents need to answer from your code and its history, the program is open. DeployIt is free during early access; paid plans arrive in January 2027.

Ready to see what your team shipped?

Continue reading