Can You Trust AI-Generated Code? Seven of My Seven Scripts Had a Fault Before They Ran

You can trust AI-generated code about as far as someone who did not write it has checked it. I have a count from my own systems that shows what the checking finds. Between the late afternoon of October 9 and the morning of October 10, 2026, an AI session wrote or rewrote seven scripts for my business, and each script changes something on a system that is running. A script is a short program that carries out a job without a person typing each step, and a session is one working run of an AI model with its own memory of what it has seen. An independent reviewer read each of the seven scripts before that script was run for real. Five reviewers handed back fourteen reports in all, and all seven scripts had at least one fault.

The five reviewers were AI too. Each reviewer was a fresh session on the same model that wrote the scripts, handed the files and told to change nothing. Keeping AI's work checked before it touches anything live is part of the AI and data work I do for small businesses at Pioneer, so my own systems were the first to get that check.

AI-generated code almost always compiles, and compiling is not the same as being right

AI-generated code almost always compiles, meaning a computer will accept the code as a valid program, which is why owners trust it too early. Veracode, a company that sells code security testing, published the 2026 edition of its report on AI-written code on July 28, 2026. Veracode's report says "models generate compilable code at a near-universal syntax pass rate of ~100 percent." Veracode's 2026 report also says "the average security pass rate sits at 56 percent" across more than 100 models tracked since the program began.

Newer and bigger AI models have not closed the gap between code that compiles and code that is secure. Veracode's 2026 release says "model size has no impact on security performance," and the best model in the 2026 report passed 68 percent of the security tests.

Developers report more distrust than trust in what AI tools produce. In Stack Overflow's 2025 developer survey, 46 percent of respondents said they distrust the accuracy of AI tools and 33 percent said they trust it. Sonar, which sells code-checking software, surveyed more than 1,100 developers and published the results in January 2026. Sonar reported that 96 percent "do not fully trust that AI-generated code is functionally correct," while "only 48% state they always check their AI-assisted code before committing it."

The Veracode, Stack Overflow and Sonar figures all describe other people's code, and two of those three sources sell the check. The search results for whether you can trust AI-generated code did not turn up a page where someone counted the reviews of their own AI-written code and published what came back.

Five reviewers handed back fourteen reports, and all seven scripts had a fault

Five reviewers handed back fourteen reports between the evening of October 9 and the morning of October 10, 2026, and all seven scripts had at least one fault found before the reviewed version first ran. The first report came back at 5:58 PM and the last at 9:51 AM. The figures below were counted on October 12 from the saved records of the writing session and the five reviewer sessions.

The count of seven scripts treats one script and its installer as a single script. Two of the fourteen reports cover the settings file, which changes nothing while it is switched off, and the other twelve reports cover the seven scripts. A blocker is a fault serious enough to stop a run, and a nit is a small point. Points raised without a number, the 13-in-90 fault in the table further down among them, are not in the 129.

Each reviewer was a separate AI session started with a clean slate. All five review requests opened the same way: "You are an independent reviewer. You did not write" the code in question. A reviewer could read the files and run tests on scratch copies of its own. No reviewer was allowed to change a file or to contact a live system. The writing session and all five reviewers ran on Claude Opus 5.5, a model in Anthropic's current lineup.

The five reviewers did more than read. The reviewers made 542 calls to a command line, and the fourteen reports describe what many of those calls were for: rebuilding a file in a scratch folder, running one check 800 times, walking one script through 27 runs.

The count of 129 findings deserves some suspicion, and the company that makes the reviewers says why. Anthropic's documentation for its coding tool warns that "A reviewer prompted to find gaps will usually report some, even when the work is sound, because that is what it was asked to do." The count of 129 puts nits and notes beside blockers. The reviewers' verdicts are stronger evidence than the count of findings. Four of the seven scripts came back at least once with a plain no, in words like "do not run until 1 and 2 are fixed" and "No, not as it stands." The other three scripts drew no blocker. The faults found in those three were filed as things to fix or as improvements, and those three are the scripts that ran on October 10.

One reviewer read the same script four times. That reviewer's first three reports numbered 20, 16 and 7 findings. The fourth report numbered none, left one small nit, and gave its verdict in seven words: "Install it. Nothing has to change first."

The clearest fault was a safety check that looked right and failed open

The clearest fault was a safety check that let through the one thing the check existed to stop. One of the seven scripts deletes something that is no longer needed. Before deleting, the script checked that nothing still in use mentioned the thing being deleted. The safety check read as correct.

The safety check piped a long settings file into a search. A pipe hands one program's output to the next program. The search was told to stop at its first match, which is the efficient way to ask whether a word appears at all. The script also ran with a strict setting turned on in the shell, the program that runs a script's commands. Careful script writers turn that strict setting on so that a failure anywhere in a pipe counts as a failure of the whole pipe.

The early-stopping search and the strict setting cancel each other out. The manual for the search tool says a quiet search will "Exit immediately with zero status if any match is found, even if an error was detected." When the search quits early, the program still feeding the file into the pipe is cut off partway and counts as failed. The shell's manual says the result of a pipe under the strict setting is the result of the last command in the pipe that failed. So the whole pipe read as failed, the script read a failed search as "not found," and the script carried on to the delete.

The reviewer rebuilt the settings file with the old item still named in it and ran the script's check word for word 800 times, using the tools on my Mac. The check let the delete through 790 times. Nobody had run the delete script yet, and the reviewer could not test the tools on the machine where the delete script would run. A fair caveat came with the finding: "As of yesterday's inventory it would do no harm; the safety net is what is broken."

The fault reproduces on a file that has nothing to do with my systems. On October 12, 2026 the same shape of check was run 200 times against a made-up file of 1.4 megabytes that names the item on its third line.

• With the strict setting on, the piped check said "not found" 200 times out of 200.
• With the strict setting off, the same piped check said "not found" 0 times out of 200.
• A version that reads the whole file first and then counts the matches said "not found" 0 times out of 200.

File size decides whether the fault shows, which is how a check like this one survives its author's tests. The same piped check was run against made-up files of seven sizes, six times over. Up to 60 kilobytes, the piped check was right in all 200 trials of every run. At 72 kilobytes the piped check flipped and was wrong between 192 and 200 times out of 200. From 144 kilobytes up, the piped check was wrong 200 times out of 200 in every run. A small test file, on my Mac at least, passes every time.

The fix for the safety check reads the whole file first and then counts the matches. The reviewer's corrected line refused the delete 800 times out of 800, and the reworked check refused 200 times out of 200 in the writing session's own test.

Four kinds of fault kept coming back, and all four are about the bad day

Four kinds of fault came up more than once in the fourteen reports, and all four concern what a script does when a step fails, the script is cut off, or a person presses stop.

The undo faults go to the heart of one script. That script makes a change and undoes the change when the script cannot prove the change is safe. A script built that way is only as safe as its undo step, and the first version's undo step never started if the terminal window was closed. A second press of Ctrl-C, the keyboard's stop command, could end the undo step too, because Ctrl-C reaches every program the script has running in that window and not only the script.

The reviewers also checked the evidence the writing session leaned on. One saved answer did not record which system the question had been put to, so the answer could not be used, and the question was asked again the next morning.

The reviews had faults too, and the rehearsals hid some

The reviews were not clean, and the reports say so themselves. Four times a reviewer wrote that an earlier fix was wrong, and other fixes came back marked as partly done. One of the wrong fixes was the reviewer's own advice, and the report says so in five words: "'Log first' was my advice." Three more times a reviewer wrote that its own earlier pass had fallen short, once in the words "I missed it last time."

Five of the seven scripts also have a rehearsal, which is a practice run against stand-in programs that contacts nothing real. A stand-in is a fake version of a real tool that answers the way the real tool is expected to answer. By the end of the morning of October 10 the five rehearsals held 271 checks, and my notes from that morning record every check passing. A passing rehearsal proved less than it appeared to.

• A stand-in was too kind. One stand-in answered a start command at once. The real command waits for the whole job to finish, so a time limit written into the script never got a turn.
• One rehearsal did not test four of the fixes. That rehearsal passed 47 of 47 checks with each of the four fixes taken out, one at a time. The rehearsal was then changed so that removing any one of the four fixes makes the rehearsal fail.
• The test code itself could do damage. A reviewer showed that one script's test could delete the folder the test was run from.

The rule that came out of the morning of October 10 is short. A new check counts only after the check has been shown to fail against a copy without the fix.

One script that went through four reviews was 195 lines long at its first review and is 391 lines long now.

The same model wrote the faults and found them

A newer model is not what separated the faults from the findings, because one model did both. The writing session and all five reviewers ran on Claude Opus 5.5, which Anthropic's model guide lists in its current lineup and names as the model to start with for most work. What separated the writer from the reviewers was what each one had in front of it. The writer was working from its own memory of the job. Each reviewer started with the files, the facts and a written request, and none of the writer's memory of the job.

Anthropic's documentation for its coding tool describes the effect of a clean slate: "A fresh context improves code review since Claude won't be biased toward code it just wrote." Researchers have measured an AI model's bias toward its own work from the other direction. A 2024 paper on AI models used as graders describes "self-preference, where an LLM evaluator scores its own outputs higher than others'" even when human graders rate the outputs as equal in quality. LLM stands for large language model, the kind of AI behind chat assistants.

My blog drafts gave the same result before my code did. The fact check on those blog drafts found something wrong in nine drafts out of nine, and each of the nine drafts had already passed two other full reads.

Four limits belong on the count in this post.

1. No person read these scripts line by line, and that includes me. The session that wrote the scripts read the fourteen reports and made the fixes. My part was the approval, the questions the session brought back to me, and the keyboard for the four scripts held back for me to run.
2. The reviewers started from the writer's description of the job. The writing session wrote every review request, and two of the requests carried an earlier reviewer's findings.
3. A fault found is not harm prevented. The reviewer of the delete script said that, as things stood the day before the review, the delete script would have done no harm.
4. There is no comparison group. None of the seven scripts was run without a review to show what would have happened.

Four of the seven scripts have run, and one waited almost 38 hours at its own question

Four of the seven scripts have run as of October 12, 2026, and each of the four made its change only after a reviewer cleared it. Three ran on the morning of October 10, after last words such as "Install it. Nothing has to change first." Two of those three were edited once more after the reviewer's last look, each time to apply a fix that reviewer had asked for. One of the three was also run three times in a look-only mode on the real system between reviews, each time before the reviewer had seen that version, and the first of those look-only runs left an empty file behind. I had approved the recommended actions in advance that morning and left my desk, which is a decision about how much authority to hand over and not a review. One reviewer would not stretch that approval to cover the script in front of it. That reviewer's verdict was "no, not unattended," and one of the reviewer's reasons was that my own written rules ask for "an explicit yes per live change, which a general approval relayed to me is not." The script that reviewer refused had still not run when its state was last read, on the morning of October 12.

The fourth script ran on October 12, with one line changed at its final reviewer's request. On its first look, a reviewer had flagged a point about waiting: the script read one of its safety conditions once, before asking its yes-or-no question, and the question could wait any length of time. The fix makes the script read that condition again after the answer. On the script's first real run, the question waited from Saturday evening to Monday morning, just under 38 hours. Nobody can say the yes was rushed. The second read found nothing to stop, and what the first real run showed is that the reviewer's premise was real. The script then made its change, proved the change against a run it had started itself, and finished about eight minutes after the yes.

The other three scripts are still on my list.

Three questions to ask before AI-generated code touches anything live

Three questions settle most of whether to trust a piece of AI-generated code, and an owner who cannot read code can ask all three.

1. Who reviewed it that did not write it? A second session with a clean slate counts. The author reading its own work again does not. The rule that an author's work needs a second reader is older than AI. In 2012 the trading firm Knight Capital put new code on seven of its eight servers and missed the eighth. The Securities and Exchange Commission's order says Knight Capital "did not have a second technician review this deployment" and lost more than $460 million on trades the eighth server sent out over about 45 minutes.
2. What was it rehearsed against, and has the rehearsal ever failed? A practice run that has only ever passed proves little. Ask to see the check fail against the old version.
3. What does it do when something fails halfway? Ask what the script prints, and what state the script leaves behind, when a step fails or someone presses stop in the middle. Every row in the table of faults above came from that question.

My own rule is that AI proposes and a human decides, and for code the review is what makes the deciding possible. Seven scripts, at least one fault in each, and under five and a half hours of reviewer time for everything that was read. I would not run an eighth script without that review.