An employee has built a tool with AI: how can you keep it maintainable?

How can you keep a tool built with AI by an employee maintainable?

A tool built with AI by an employee remains maintainable with five habits from the outset. Those habits are a repository with its history, a test copy separate from production, tests for every change, explicit permissions and gateways, and dated decisions. Include these requirements in the operating method that goes with the tool. They are part of the work that you ask the employee to do. They remain part of that work even when the first version meets a very limited need. Ask for accessible evidence. A verbal assurance is not sufficient for you to check that these habits exist.

1. Keep the code in a repository with its history. Ask for the location of the version that is in use. Check that the repository keeps the history of changes. The company must have access to it through an account that it controls.
2. Provide a separate test copy. Have the environment for testing and the environment for everyday users clearly identified. The employee must show you where they will check a change before it goes into actual use.
3. Require tests for every change. For each proposed development, have the expected behavior and planned checks specified. Keep the results of the checks with the relevant change, including when a check fails.
4. Make permissions and gateways explicit. Have a list kept of the access granted to the tool and its connections to external services. Each connection must have a stated use and a person responsible for it.
5. Record decisions with their dates. Ask for a note when a choice has implications for the work that follows. It must specify the reason for the choice and the constraints known at the time. A few lines may suffice.

These are five points that you can check to guide the developments. If a point is missing, add its implementation to the work that is planned. Ask for a specific deliverable: you can check access to the repository, for example, by opening it.

Why does the tenth agent break the work that the first had managed to get right?

The tenth agent receives the code without the reason for the choices that were made initially. It can then break the functionality that was working. The risk is to remove a precaution by mistaking it for an error. Consider a tool that excludes the records with no email address. The filter keeps follow-ups that cannot be sent from being attempted. An agent has to retrieve the customers that are missing from the table. It removes the filter, and the follow-ups fail. Another agent restores the filter to correct the sending of follow-ups. The customers disappear from the table that lists them again. The fix goes around in circles: nobody has specified to the agent that the two uses have different needs.

In a randomized trial involving 16 experienced developers working on 246 real tasks from their own repositories, the use of AI tools increases completion time by 19%, although the developers subsequently estimate that they were 20% faster.

Across twelve months of data from real repositories, the share of changed lines associated with refactoring falls from 25% in 2021 to less than 10% in 2024, while copied and pasted code increases fourfold and exceeds moved code for the first time.

In the 2025 developer survey, 84% use or plan to use AI tools (76% the previous year), more distrust the accuracy of these tools (46%) than trust it (33%), and 66% cite solutions that are almost right, but not quite, as their first frustration.

How do we organize the maintenance of our own system?

We organize our maintenance around modules that are frozen with a certificate. Each change follows a planned process. Written and dated decisions accompany the work that we do. A test mirror allows you to perform the checks that are needed. We keep a test for each defect that we have fixed. The freeze imposes a stop. Does an agent propose to simplify the code during an intervention that concerns something else? That does not justify modifying a part that has already been validated in the process. Our operating method sets the boundary for that. Modifying a frozen module is an intervention of its own that has to be treated as such. There must be an explicit reason, followed by another check before the part can be put back into service.

The 2025 DORA report observes a positive relationship between AI adoption and software delivery throughput, but a relationship that remains negative between AI adoption and delivery stability; AI does not fix a team, it amplifies what already exists.

The certificate must specify the scope that was validated and the conditions of that validation. The person responsible for the next intervention must know what they can rely on. They must also know what the change that they make requires them to check again. To assess the arrangement, check whether the validation still corresponds to the version that is in use. A certificate that is attached to an old version is not sufficient. It does not establish that the modified version works under the same conditions.

To apply this arrangement to your tool, ask the person responsible to prepare the next intervention. They must identify the module concerned and the decision that authorizes its reopening. That precedes the modification work that is assigned to an agent. Have the intervention go through the test mirror that is provided for in the maintenance process. Then ask for the defect that has been fixed to leave a test in the project. That record is part of the work delivered. At validation, require a dated decision and a certificate that corresponds to the proposed version. Have the checks that allow the module to be closed again specified. Also have it specified who takes responsibility for putting the module back into service.

What happens to the connection when a provider changes its rules or access?

A provider changes the rules or access that it has. The connection can then fail. Each gateway must have somebody responsible for adapting it to the service’s new conditions. Consider a tool that retrieves the orders from a provider to prepare customer follow-up. The provider requires new authorization. The person responsible for this connection must be able to find the data received and transmitted by the tool. They must also be able to find the permissions of the account that is used for this exchange. Access to view the orders does not necessarily give the right to modify them. The record for the gateway must specify that limit. Record the name of the person responsible for adaptation in your operating method. Specify who in the company can authorize a change of permissions. If the provider requests broader access, the employee responsible for repairing the connection must know to whom they should submit the request. They must know to whom they should submit the request before accepting it.

What should you write so that another person can take over the tool?

To prepare for the handover, keep the history of changes to the tool in the repository. Link significant changes to a dated decision that explains the reason for them. The history allows you to find successive versions of the code and their differences. The decision text specifies the need that the choice was intended to meet. It also keeps a record of the constraints that were known at the time. The person taking over the tool must be able to start from a change visible in the history and find its explanation. They must be able to do that without asking the employee to reconstruct an old discussion from memory.

For each choice that affects subsequent work, ask for a note kept with the code. Ask for the date and the name of the person who decided. Then ask for the problem encountered and the reason for the solution selected. Also keep the approach that was rejected when it explains a limitation that is still present in the tool. Specify the reason that led to its rejection. Specify the circumstances in which it could become relevant again for the person responsible for taking over. A few lines may suffice. “Change to simplify” leaves the reader to guess what it was that caused a problem. Describe the concrete difficulty that the change was intended to resolve. Then associate the note with the change that it concerns. Its explanation must remain accessible from the repository.

If a decision changes, keep the initial text of that decision. Add a new note that explicitly identifies the note that it replaces. Specify the reason for the change and its date. The person taking over the work will be able to distinguish a constraint that remains valid from a choice that was abandoned. They will be able to make that distinction even when consulting an old version to understand the tool’s operation at that time. Include a handover check in the operating method that you use. Ask the person taking over the tool to find the reason for a significant change from the repository alone. Then supplement the note with the explanations that they still had to request verbally.

Why should you test a change on a copy that is separate from production?

A copy separate from production allows you to check a change without interrupting the users’ work. There is one condition: the copy must not modify their data or cause real actions to be triggered. Consider a change to the calculation of discounts in a tool for preparing quotes. The person responsible for the change installs it on a test version. They use fictitious quotes for which the amounts that should result are known to them. They compare the results obtained with the results that were expected before proposing use by the sales team. The separation must be real. Does the copy still use the file of orders that belongs to the company? Can it send a quote to a customer? The test can then have consequences in production. Give the copy data of its own. Direct any messages that it sends to a mailbox reserved for testing. Direct its messages to that mailbox even if the screen already says “test”.

Set a condition for moving to the version that the team uses. The person who knows the work concerned must try the modified workflow on this copy, through to the final document that it produces. They must confirm that the result corresponds to the requested need. For the discount calculation, have them go through an ordinary quote and then a quote with an exceptional discount. Ask them to check the amount on the document that is intended for the customer. The preview in the tool is not sufficient for that. If a discrepancy remains unexplained, keep the change on the copy that you use for testing. Continue to check it there. The users continue their work on the version that is in service for the duration of this review.

How can you prevent a fixed defect from returning with the next change?

To detect the return of a fixed defect, turn the case that revealed it into an automated test. Require the test to pass before you deploy to production. A defect that returns after a change is given the name of a regression. Consider a fictitious example: a tracking tool creates a duplicate order when an employee imports the same file twice into it. After the fix, the second import must leave the list of orders as it was. That expected behavior becomes the rule to check. It remains the rule to check even if the next change concerns another function of the tool.

1. Reproduce the defect by testing for it. Prepare a fictitious file that has an order as its contents. Have the test carry out two imports that follow each other. Ask it to check that only one order exists at the end. The result that is expected must correspond to the needs of the employee.
2. Check that the test fails before you apply the fix. This step matters. A test that passes while the duplicate exists does not detect the problem. Then apply the fix and rerun the test with exactly the same setup. It must now pass, without relaxing the result that you expect.
3. Run this test for every change that you make. Configure continuous integration for that. It automatically runs the checks with each new change that is submitted to the repository. Keep the test for the double import with these checks. Keep it with those checks even when nobody believes they have touched the order processing.
4. Block deployment to production if the test does not pass. A person can ignore the alert by itself. Configure the delivery process so that a failed result will block publication. Then examine the change that caused the duplicate to return.

A passing test confirms only the behavior that it checks. If another defect appears, add the case that corresponds to it. The operating method thus keeps an executable check of each fix that has been made. The next person to modify the tool must have access to it.

When should you organize maintenance of the tool built with AI by your employee?

Organize maintenance as soon as a second person depends on the tool built with AI by your employee or needs to modify it. At that threshold, designate the person who will be responsible. They must check that the five habits exist before committing to the next development. They have to perform that check even if the tool is satisfactory today. Make that check a condition for authorizing the work to continue.