What happens in the first week an AI agent is connected to your product?

Three things happen, rarely in the order you expect: questions the code answers, questions that miss because the answer isn't in the code, and questions left without an answer. The first week is for sorting them, not for scoring the agent.

This article walks through those seven days for an agent connected to a private product through an MCP server. There is no real usage figure we could cite, so the scenario below is illustrative: the company and its questions are fictional.

What "connected" means on day 1

A connected agent is an agent that can call a source at the moment of the question. It doesn't know the source by heart; it consults it. With DeployIt, that source is one MCP server per project, read-only, revocable at any time, with content resynchronized on every push. The agent calling it holds no credential on your repository.

That sets what day 1 allows: questions whose answer is in the code or its history, and nothing more. The MCP server is in early access, through a design partner program; what you observe in week one is part of what we want to learn with you.

The scenario: a question log, sorted into three piles

Take a small fictional team that connects a support agent to its product. It logs every question asked during the week and files each into one of three piles. A few lines of that log (fictional exchange, labeled illustrative):

None of these rows is a model bug. Each one says something about the boundary between what the code establishes and what it leaves open.

Pile 1: what lands

These are questions of fact: a behavior, a default value, a recent change. The answer exists in the code, the agent reads it at question time, and cites it. The flagship question, “What changed in the product this week?”, is one of them: it reads from the diffs of the window.

One sign to check from day one: the answer names its source. A correct answer without a source can't be told apart from a lucky one.

Pile 2: what misses

This is the most useful pile to read, because it's the one that costs when it reaches a customer. It holds the questions whose answer isn't in the code, or is there in a confusing form: a commercial policy, a service commitment, a word that means one thing to customers and another in the code.

The mechanism is the one described in agents that invent features: the tool is called, the source returns something adjacent, and the agent stitches. The remedy isn't a bigger model. It's knowing, question by question, where the answer should live: if it's in a hand-written document, that document is what the agent needs, with its own age.

Pile 3: what stays unanswered

A roadmap date, a decision not yet made, a detail nobody wrote down: the correct answer is "that isn't in what I can read". Silence is a result, not an outage, provided it is stated.

Look hardest at the day a silence was filled. If the agent answered question 5 with a date, nothing in its setup tells it to stay quiet, and that is the first thing to fix.

What you adjust, and on which side

Sorting only matters if it leads to changes. Four are within your reach, and only four:

1. Which questions go to the agent. Questions of fact, yes. Commercial or roadmap questions go to the person or document that holds them.
2. The silence instruction. Tell the agent to write "I can't find it" instead of filling in, and check the instruction holds on three real cases.
3. The vocabulary. If the product and the code use different words (row 6 of the table), write the mapping once in the agent's instructions.
4. The source for answers outside the code. A hand-written document ages: the agent will read it as is, at whatever age it has.

What our public pages don't say is how far DeployIt itself exposes settings on these points. If that is what matters to you, put the question to the design partner program: we won't present as settled what we haven't written down.

"I'll build a test set before connecting"

A good idea, and not enough. An invented test set reflects what its author already knows to ask. The questions that hurt are the ones customers ask in their own words. Keep the test set, but also read the real questions of week one, in the words of the people asking.

"My support tool already does this with my knowledge base"

It does it for what the knowledge base contains, at the age the base has. If what matters to you is the Monday question after a release, see why an agent answers wrong after a release. The two sources don't exclude each other: the hand-written base for what the code doesn't say, the MCP server for what it does.

Mini-FAQ

How many questions do you need to read to judge? Enough to see each pile represented at least three times. While a pile is empty, you haven't yet asked the questions that fill it.

What do you do with a right answer that has no source? Treat it as unverified. It may be accurate; you can't know without the source.

Should you warn the support team? Yes, before day 1: the three piles exist with humans too. The difference is that the agent doesn't flag on its own when it hesitates.

The week teaches you where the code stops

After seven days you don't know whether the agent is "good". You know which questions belong to it, which go elsewhere, and what it should say when it doesn't know. DeployIt's MCP server is in early access, through a design partner program; it is free during early access, and paid plans arrive in January 2027. For how agents consult your product without touching your code, see also giving agents access safely.