Repair, refactor or rebuild your AI-built tool?

A working prototype can have messy code and still contain valuable decisions. It may capture a workflow that nobody managed to explain in a specification. Conversely, a polished interface can conceal a design that cannot enforce the access rules your business needs. Neither appearance nor the use of AI tells you whether to repair, refactor or rebuild.

Start by separating the business behaviour you want to preserve from the implementation you are willing to change. The question is not whether a developer would have written the code differently. It is whether the current implementation can meet your requirements at a reasonable cost and level of risk.

Name the problem before choosing the treatment

“Clean up the vibe code” is too vague to estimate. Replace it with observable problems: users see records they should not; deployments require manual edits; one change breaks several screens; a report times out; nobody can restore the database; or the application cannot run outside its original builder’s account.

For each problem, record the affected workflow, business consequence, frequency and evidence. A reproducible permission bypass is different from an unfamiliar folder structure. A slow administrative report is different from a corrupted order. This prevents style preferences from displacing more consequential work.

Also record what works. Keep representative inputs and expected outputs, screenshots of important states, examples of calculations and the rules behind approval decisions. These become a baseline for checking changes. Without a baseline, a rewrite can quietly lose the behaviour that made the prototype useful.

Repair when the fault is bounded

A repair changes a specific behaviour with limited surrounding disruption. Examples include checking authorization on a server endpoint, adding a missing validation rule, correcting a calculation, or making an external request idempotent.

Repair is attractive when the cause is understood and the surrounding design supports a safe fix. It still needs verification. A permission check that protects one screen but leaves the export endpoint open is an incomplete repair. A retry that duplicates an external action can turn a transient failure into a business incident.

Define an acceptance example before implementation. For a private record, an owner may read it, a permitted colleague may read it, and an unrelated user must be rejected even when calling the endpoint directly. The value lies in proving the intended behaviour, not merely adding code that looks reassuring.

Refactor when change itself is becoming risky

Refactoring changes internal structure while aiming to preserve externally observable behaviour. It can separate business rules from presentation, consolidate repeated authorization logic, make configuration explicit or remove a dependency on one deployment environment.

The purpose should be connected to an upcoming need. If three copies of a calculation keep diverging, a shared implementation can reduce inconsistency. If the application needs a second workspace, making data boundaries explicit may be necessary before that expansion. “Make it cleaner” alone does not define a useful stopping point.

Work in increments. Protect a critical path with focused tests, move one responsibility, compare results and deploy a reversible change. A long branch containing unrelated improvements makes it difficult to understand regressions and postpone less valuable work. Small, verified changes are easier for a business owner to evaluate.

Rebuild when the foundation blocks the requirement

A rebuild may be justified if the application cannot enforce required data boundaries, depends on an unsupported environment, lacks a recoverable source, or has accumulated constraints that make the required changes more expensive than replacing a bounded component.

But rebuilding has migration costs. Existing records need a destination, identifiers may be referenced elsewhere, users need continuity, and external integrations must not run twice. There is also a period in which the old and new systems may both require attention. A fresh codebase does not make those obligations disappear.

Consider replacing one subsystem before replacing everything. A fragile reporting job may be separated from an otherwise useful interface. Authentication can sometimes be moved behind a supported identity system while preserving the application’s business workflow. The right boundary depends on the actual code and data.

Use evidence rather than an AI label

Research does not support a universal productivity promise for every tool, task and developer. METR’s update on measuring AI-assisted development discusses important measurement challenges, including selection effects. Its findings should not be turned into a prediction of your project’s speed or quality.

Likewise, a study of one coding assistant or one group of tasks cannot certify your application. We assess observable properties: reproducible builds, access control, data handling, error behaviour, tests for important rules and the ability to recover. This works whether code was written manually, generated by a model or assembled from several sources.

The OWASP Application Security Verification Standard provides a structured reference for security requirements. A scoped review may use relevant requirements as prompts. That is different from verifying the entire standard or claiming certification.

Compare options with a short decision record

For each option, write down what changes, what stays, known risks, unknowns, expected effort and how you will decide it worked. Separate an estimate supported by inspected code from a rough assumption. Identify work needed before a reliable estimate is possible.

Include the cost of doing nothing for the next few weeks. Sometimes a temporary restriction—such as keeping the tool to a small trusted group—creates time to fix a difficult boundary. Sometimes continued use is inappropriate because a serious data exposure is already reproducible. The decision should follow the evidence and consequences.

Avoid a false choice between “ship immediately” and “rebuild everything”. A useful sequence may be to repair a blocking defect, establish recovery, refactor the part that impedes necessary changes, and leave harmless imperfections alone.

Agree on a stopping point

A repair ends when the specified fault is resolved and its important adjacent paths are checked. A refactor ends when the intended responsibility is clearer and behaviour remains verified. A rebuild needs explicit migration, cutover, rollback and retirement criteria.

Ask for a recommendation that distinguishes blockers from improvements and identifies what was not inspected. That makes the next investment reviewable. A technical opinion is most useful when it explains why one action matters now and another can wait.

Our Starter Review is designed to produce that kind of bounded next-step assessment. It includes up to four total engineering hours for inspection, report and walkthrough. Repairs, deeper investigations and implementation are separately scoped; the review is not a promise to make an entire application production-ready.