AI-Generated Code: Who Still Understands Your Software?

A developer sits down at a keyboard, opens a forum and starts typing. He gives neither his name nor his employer’s. In a few paragraphs, he tells the story of an entire year of his working life. His post, published on Reddit and reposted on Lemmy, has since been picked up in France under a headline that leaves no room for doubt: an opaque labyrinth (Developpez.com).

The story starts like a success story. A year ago, leadership announced that all development would now go through AI agents. Writing code by hand became the exception, and the exception had to be justified. For six months, the team lived through a kind of euphoria: it shipped more than it had ever shipped (Lemmy).

Then the picture changed. The codebase has roughly tripled in size. Nobody can explain how most of it works anymore. The agents themselves get lost: they keep re-implementing helper functions that already exist somewhere else, because there are now four versions of everything (Lemmy).

I owe you a caveat before going further. This testimony is anonymous. Nobody can verify the company’s name or the tripling figure. So I treat it for what it is: one person’s account, with its biases. It still deserves your attention, because every symptom it describes has been measured elsewhere, by researchers who publish their methods and their data.

Six months of euphoria, then the bill

Let’s start with the euphoria. It has an explanation, and it can be measured.

In 2025, the research organization METR ran a randomized controlled trial with 16 experienced developers. They worked through 246 real tasks on open-source projects they had known for years, codebases often exceeding a million lines. Each task was randomly assigned: with AI or without. Before the experiment, the developers expected to save 24% of their time. Afterwards, they believed they had saved 20%. The stopwatch said otherwise: with AI, they had taken 19% longer (METR).

Read those three numbers again. Seasoned professionals, working on their own code, felt faster while they were slower. Perception and measurement pointed in opposite directions.

The study’s authors stay cautious, and I appreciate that caution: their results cover a specific context and do not prove that AI slows everyone down, everywhere. But they show something no executive should ignore. The feeling of speed remains an assumption until you measure it. And assumptions must be killed with experimentation as early as possible.

The team in the testimony lived exactly this gap. It measured what was easy to measure, the volume shipped, and it mistook that volume for value.

The second symptom, code that swells and repeats itself, has been measured too. The company GitClear analyzed 211 million changed lines of code between 2020 and 2024. In 2024, the frequency of duplicated blocks of five lines or more rose eightfold. That same year, for the first time, copy-pasted code exceeded moved code, meaning code reorganized so it can be reused (GitClear).

Translate that result into boardroom language: your software is growing faster than it is getting organized. Every duplicate is one more place where a fix will have to be applied, and one more place where someone will forget to apply it. The “four versions of everything” in the testimony look a lot like what GitClear observes at scale.

A program lives in the heads of the people who built it

To understand why nobody can explain this code anymore, we need to go back to 1985. That year, Peter Naur, Turing Award winner and co-author of the BNF notation, published a short essay called “Programming as Theory Building.” His thesis is still unsettling today (Peter Naur).

For Naur, programming is first of all building a theory: an understanding of the problem and of the reasons behind each choice. The code is only a partial trace of that theory. The theory itself lives in the heads of the people who built the program. Naur goes further. In his view, rebuilding the theory of a program from its documentation alone is strictly impossible. A program dies when the team that holds its theory breaks up, and its death becomes visible the day nobody can intelligently answer a request to modify it (Peter Naur).

Read the testimony again through that lens. For a year, the company produced code without anyone building the matching theory. The agent generates, the test passes. Nobody had to understand. The code exists, and the theory exists nowhere.

Every CTO knows the moment when the architect who carried a system in their head leaves the company. I led more than 1,100 engineers, and that kind of departure is one of the risks you watch. Here, the situation is even more worrying. Nobody left with the theory, because it was never built.

Two reviewers leave, and nobody sees the signal

One detail in the testimony struck me more than the tripling of the codebase. The two people who read every change and dared to push back left in the spring. One quit. The other was “restructured,” to use the now-standard euphemism (Lemmy).

An automated review tool still comments on every change. But nobody is left who can tell whether a comment matters. So the team marks every comment as resolved, without reading it. And the author adds a sentence that should keep any engineering leader awake at night: half the team hasn’t written anything non-trivial in a year, and it shows (Lemmy).

This passage describes a culture issue far more than a technology issue. Those two reviewers were the organization’s critical mind. They embodied the possibility of saying “I don’t understand” and “this is wrong.” Without psychological safety, nobody dares to challenge the status quo, or to admit they no longer understand what they approve. Feeling safe enough not to hide disappointing results is one of the conditions that allow a team to learn (My book, chapter 9).

This book is for CEOs, managers, and innovators ready to challenge everything and create exponential growth.

Be the first to get the english version!

Ask yourself honestly. In your organization, can a developer say in a meeting “I can no longer explain this module” without fearing for their job? If the answer is no, your labyrinth may already be under construction. You simply won’t see it until it is too late.

“Almost right” code is expensive

You could object that all this is about developer comfort. The code runs, customers are served, so where is the problem? It sits in what nobody looks at.

In July 2025, Veracode published a study covering 80 coding tasks given to more than 100 large language models. In 45% of cases, the code introduced a security flaw listed in the OWASP Top 10, the reference list of the most critical risks for web applications. In Java, the failure rate exceeded 70%. And security performance did not improve with newer models (Veracode).

Developers know it. Stack Overflow’s annual survey, with more than 48,000 respondents in 2025, shows that 84% of them use or plan to use AI tools. Yet 46% say they do not trust the accuracy of the output, up from 31% a year earlier. Their top frustration, cited by 66%: solutions that are “almost right, but not quite” (TechRepublic).

Combine those data with the testimony. Code where 45% of the samples tested by Veracode contained a flaw, which developers themselves often judge “almost right,” approved by a team that can no longer explain it and has stopped reading the alerts. The issue goes far beyond productivity: a business risk is growing in silence.

The metric your dashboard doesn’t show

Don’t misread me. I am not asking you to ban AI from software development. That would be absurd, and your competitors won’t wait for you. As early as Y Combinator’s winter 2025 batch, a quarter of the startups had codebases that were 95% AI-generated. One of the accelerator’s partners added that one skill would remain essential for founders: being able to read the code and find the bugs (TechCrunch).

That says it all. The mistake made by the company in the testimony lies in how it steered the change. It imposed a percentage, 100%, instead of setting a goal. It measured volume instead of understanding. And it let go of the people who held the theory of the system.

In my AI adoption method, the exploration team has to ask two questions for every use case. How can AI increase my productivity and the quality of what I deliver? What can I do now with AI that was impossible before? Any use of AI that doesn’t answer one of these two questions is a distraction. And to check that the expected gains are real, you have to measure before, during and after (My book, chapter 14).

Applied to code, this discipline gives you simple rules:

  • For each critical module, name at least one person able to explain it and change it with confidence. If nobody fills that role, you have found your risk zone.
  • Before merging any generated code, ask its author to explain in their own words what the code does and why. If they can’t, the code isn’t ready.
  • Keep humans on the non-trivial parts. A skill you stop practicing withers, and the testimony shows it.
  • Measure duplication and security flaws in addition to the volume shipped.

None of this forbids AI. These rules only prevent AI from producing a codebase with no owner.

The essentials

Remember four things:

  • The feeling of going faster remains an assumption to verify. In the METR study, experienced developers believed they were 20% faster while they were 19% slower.
  • A program lives in the heads of the people who understand it. Code that nobody has understood is a program without a theory, and therefore without a future.
  • Reviewers who say no are an asset. When they leave, your organization loses its ability to see its own mistakes.
  • Set goals for understanding. A percentage of generated code says nothing about the value produced. Name a human owner for each critical module.

To get out of the labyrinth, the company in the testimony will have to rebuild the theory of its own software. That will take time and the humility to admit that the speed it felt was an illusion. A company that doesn’t innovate is already dead. A company that confuses volume with innovation may find out later than the others. Innovate or agonize, the choice is up to you!

References

Picture of Philippe Boulanger

Philippe Boulanger

Philippe Boulanger, international speaker on innovation and artificial intelligence, author, advisor, mentor and consultant.

Latest POSTS

41 Shades of Blue: Google’s $200 Million A/B Test

By testing 41 shades of blue, Google claims it earned an extra $200 million a year. Behind the cult A/B testing anecdote lies a debate every leader has to settle: when to trust the data, and when it stops you from innovating.

Read More »

Drone vs Camera Robot: Who Films Where the Sky Is Banned?

Banned from flying over crowds, drones run into regulation and noise. The Beni camera robot films from the ground. A textbook case of the framing bias.

Read More »

Friday the 13th: why superstition pays off for innovators

Friday the 13th destroys nothing. People who fear it change their behavior, and that change can be measured. Accidents, stock markets, the Jerusalem virus, a $900 million figure nobody verified: what superstition reveals about your innovation decisions, and how to use it.

Read More »

Naming an innovation: what Sapir showed in 1929

In 1929, Edward Sapir showed that most people link the invented word “mal” to a large table and “mil” to a small one. Almost a century later, sound symbolism still shapes how we perceive product names. What the numbers really say, what marketing took from it, and how to test a name before you pick it.

Read More »

Why Coca-Cola’s Inventor Never Got Rich

John Pemberton invented Coca-Cola in 1886 and died poor two years later. Thirty-three years after the first glass, the company was worth $25 million. What this story reveals about the difference between invention and innovation, and about the Pemberton who may be sitting in your own company.

Read More »

Rare-Earth-Free Motors: France Had a 14-Year Head Start

Renault has built rare-earth-free motors since 2012. Then the stator of its next E7A motor went to China. How a technological lead gets lost in purchasing trade-offs.

Read More »

Leeuwenhoek: the cloth merchant who saw microbes first

In 1676, a Delft draper with no university education discovered microbes. The Royal Society didn’t take his word for it: it checked, then elected him. What the accurate story of Leeuwenhoek teaches leaders about ideas that come from outside.

Read More »

Edison’s salt test is a legend. His 146 questions are not.

The salted soup test attributed to Edison is a legend. His 1921 questionnaire, however, failed 686 of 718 applicants, and Einstein himself stumbled on one of its questions. From Xero’s coffee cup to Musk’s riddle, what these hiring tests really measure, and what hiring science recommends instead.

Read More »

Are you a rule breaker?

You weren’t supposed to find this.

But here you are, because you did what most people don’t: you questioned, you explored, you clicked the thing you weren’t sure you should click.

That’s Innovational Intelligence™ in action.

Most people stay inside the lines. Follow the expected path. Click the obvious buttons. Accept things as they are.

Not you.

You’re one of those rare minds that refuses to accept “this is how it’s always been done.”

We need more people who think like you.

So here’s your reward for coloring outside the lines:

Get VIP pre-release access to the next assessment on Innovational Intelligence™:

You’ll be the first to know when it’s available.

Keep breaking rules. The world needs what you see.

Your daily dose (or almost) of Innovational Intelligence™!

Receive in your inbox my latest post before everyone else!