Most of what is written about AI and software development is either a sales pitch or a lament. Underneath both there is now a reasonable body of evidence: randomised trials, large-scale repository analyses and annual surveys of tens of thousands of developers. The studies we rely on when we decide how to work are worth taking one at a time, with their sample sizes and limitations attached, because the limitations are where the useful conclusions live.
Adoption is no longer the question
The 2025 Stack Overflow Developer Survey (49,000+ respondents) found that 84% of developers use or plan to use AI tools in their work, up from 76% the year before, and that 51% of professional developers use them daily.
Whatever one thinks of the tools, the professionals building software have already made the choice. The open question is how well they use them, and that is what the rest of the evidence is about.
Speed: real, but not uniform, and badly estimated
Two randomised controlled trials point in opposite directions, and both are worth taking seriously.
Google, 2024. 96 full-time engineers were randomly assigned to complete the same enterprise task, editing 10 files and 474 lines to build a logging service on Google’s internal infrastructure, with or without three AI features. The AI group finished in 96 minutes against 114 minutes, a ~21% speed-up, with a wide confidence interval.
METR, 2025. 16 experienced open-source maintainers completed 246 real tasks in their own mature repositories (average 22,000+ stars, 1M+ lines), each task randomly assigned to allow or disallow AI tools. With AI, they were 19% slower. Before the study they expected to be 24% faster; afterwards they believed they had been 20% faster. METR now labels the result as historical, since both the tools and the workflows have moved on.
- +21%
- Google RCT, enterprise task
- −19%
- METR RCT, expert maintainers
- +20%
- what those experts believed
Read together rather than as a contradiction, the two trials say something specific. Gains are large on well-scoped tasks in codebases the tool can hold in view. They shrink or reverse on large, mature systems with deep implicit context, where the expert’s own knowledge is the bottleneck and the tool’s suggestions have to be verified against it. And self-reports are unreliable in both directions, so “it feels faster” is not evidence.
Throughput up, stability down
Google’s DORA programme surveyed roughly 5,000 technology professionals for its 2025 State of AI-assisted Software Development report. Its finding: AI adoption now correlates positively with delivery throughput, and continues to correlate with higher instability, meaning more change failures and more rework.
DORA’s explanation is that AI amplifies what a team already has. Teams with strong automated testing, disciplined version control and fast feedback loops convert extra output into shipped software. Teams without those controls convert it into incidents. The report’s phrase for the second case is “accelerated chaos.”
Quality: more code, less structure
GitClear analysed 211 million changed lines of code from 2020 to 2024, a period that covers the arrival of AI assistants in mainstream use. Three trends stand out:
- Lines classified as copy-pasted rose from 8.3% to 12.3% of changes (2021 to 2024). In 2024, for the first time, duplicated lines outnumbered moved lines.
- Moved lines, GitClear’s proxy for refactoring, fell from 24.1% to 9.5% (2020 to 2024).
- Churn, code revised within two weeks of being written, rose from 3.1% to 5.7%.
This is observational data, not a controlled experiment, and GitClear sells code-analytics tooling. But the direction is consistent with what the tools optimise for: producing a working answer now, not maintaining the shape of the system over years. Duplication and reduced refactoring are exactly the costs that show up in year two, when a product needs to change.
Security: functionally correct, frequently insecure
Veracode’s GenAI Code Security research tested over 150 large language models on 80 coding tasks across four languages and four vulnerability classes. In the Spring 2026 update, models introduced a known security flaw in 45% of cases, a figure essentially unchanged since the first report. Java was the worst language at a 72% failure rate; cross-site scripting and log injection tasks failed 86% and 88% of the time.
Two details are more important than the headline. Syntactic correctness across the same models now exceeds 95%, so the code runs. And model size makes almost no difference: security performance “clusters around the same disappointing 55% mark” from 20-billion to 400-billion parameters, and the small gap there was has closed with recent releases. Waiting for a bigger model will not fix this. Review by someone who knows what an injection flaw looks like will.
Trust: the people using the tools have the clearest view
Back to the Stack Overflow survey, because the most instructive numbers are about trust rather than usage. 33% of developers trust the accuracy of AI output; 46% distrust it, up from 31% a year earlier. The single most cited frustration, at 66%, is “AI solutions that are almost right, but not quite.” 45% report that debugging AI-generated code is more time-consuming than expected. And when asked when they would still turn to another person, 75% answered: when they don’t trust the AI’s answer.
The developers with the most hands-on hours have arrived at a working model: use it constantly, trust it conditionally, and keep a competent human as the final check.
What we take from this
Putting the six sources together:
- Not using AI is a choice to be slower on most of the work. The Google trial, the DORA throughput finding and the adoption figures all point the same way for the bread-and-butter of product development.
- Using it without strong engineering controls is a choice to be less stable and less secure. DORA’s instability correlation, GitClear’s quality trends and Veracode’s 45% are the bill for skipping that.
- Expertise is the moderating variable. In every dataset, the outcome depends on who is directing the tool and what system it is embedded in, not on the tool itself.
That is why our position is the unglamorous one: AI drafts most of what we ship, and engineers who could have written it decide what ships. Architecture, security and data decisions are never delegated to a model, and every change is reviewed against an evaluation the model didn’t write. The research doesn’t say AI-assisted development is good or bad. It says it is an amplifier, and it matters a great deal what you point it at.
Sources
- Stack Overflow, 2025 Developer Survey: AI
- Paradis et al., Google, How much does AI impact development speed? An enterprise-based randomized controlled trial (2024)
- METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (2025)
- DORA / Google Cloud, State of AI-assisted Software Development 2025
- GitClear, AI Copilot Code Quality: 2025 Data Suggests 4x Growth in Code Clones
- Veracode, Spring 2026 GenAI Code Security Update and the 2025 GenAI Code Security Report