Somebody in a room of college instructors put their hand up last spring and asked whether they should let students use Artificial intelligence (AI) on a first draft. I gave them the line I had been giving for about two years. AI levels the playing field. It helps the beginner more than the expert, so yes, and the equity argument is the strongest reason to say yes.
I still think the first half of that is true. I have stopped saying the second half, and this is the edition where I explain why.
Key takeaways
- Large workplace studies show AI helps beginners most: Noy and Zhang found a 40 percent time saving across 453 professionals, and Brynjolfsson’s team found a 34 percent gain for novice support agents.
- Those studies measured performance with the tool in hand. Bastani’s classroom study measured what was left once it was taken away, and students with unrestricted AI did about 17 percent worse.
- A version that gave hints instead of answers largely removed the harm. The design choice did more work than the technology.
The evidence for levelling is strong
Start with how strong the evidence for levelling is, because it is stronger than I expected when I went looking.
Shakked Noy and Whitney Zhang ran a preregistered experiment with 453 college-educated professionals on midlevel writing tasks (Noy and Zhang, Science, 2023). Time taken fell 40 percent, quality rose 18 percent, and the gains landed unevenly in the direction you would hope: the tool compressed the productivity distribution by helping lower-ability workers more. Inequality between workers went down.
Then Erik Brynjolfsson, Danielle Li and Lindsey Raymond followed 5,179 customer support agents through a staggered rollout (Brynjolfsson, Li and Raymond, Quarterly Journal of Economics, 2025). Issues resolved per hour rose 14 percent on average, 34 percent for novice and lower-skilled agents, and only minimally for the most experienced.
Two large studies, one preregistered, one at real scale in a real workplace. If you want to argue that AI helps beginners most, you have the evidence and I am not going to pretend otherwise.
One caveat sits inside even that good news, and it matters for who we are talking about. In a field experiment with 758 consultants, Fabrizio Dell’Acqua and colleagues found the gains held on tasks the model could actually do, and reversed on one deliberately built to sit outside its capability, where people using AI did worse than people without it because they trusted confident output that was wrong (Dell’Acqua et al., Organization Science, 2026). Judging when the tool is out of its depth is itself expertise. The people getting the biggest measured lift are the ones least equipped to make that call.
| Study | Sample and design | What it found | What it measured |
|---|---|---|---|
| Noy and Zhang, Science (2023) | 453 college-educated professionals; preregistered experiment, midlevel writing tasks | Time down 40%, quality up 18%; helped lower-ability workers more | Output on the task; tool “mostly substitutes for worker effort” |
| Brynjolfsson, Li and Raymond, QJE (2025) | 5,179 customer support agents; staggered rollout | Issues per hour up 14% overall, 34% for novices, minimally for the most experienced | Output over months, not capability once the tool was withdrawn |
| Dell’Acqua et al., Organization Science (2026) | 758 consultants; field experiment | Gains on tasks inside the model’s ability; worse results on a task built outside it | Whether people trusted confident output that was wrong |
| Bastani et al., PNAS (2025) | Nearly 1,000 Turkish high school students | Better with the tool; about 17% worse once it was removed; hint version largely removed the harm | Unaided performance afterwards, the only one of the four that did |
In the support study, the most experienced agents saw small declines in quality. And in the writing experiment, Noy and Zhang report that the tool “mostly substitutes for worker effort rather than complementing worker skills.”
Substitutes for effort. Honestly, I read past that clause the first three times.
Effort is the thing that does the learning.
What expertise reversal predicts
That idea is not mine and it is not new. In a 2003 review gathering roughly a decade of their own work, Slava Kalyuga and colleagues set out what they called the expertise reversal effect (Kalyuga, Ayres, Chandler and Sweller, Educational Psychologist, 2003). Instructional supports that reduce mental load work well for novices, and as the learner’s knowledge grows those same supports become unnecessary and then actively counterproductive. Scaffolding is the frame you put up in order to build the wall, and if you leave it standing you cannot get at the wall you built.
Read the productivity studies against that review and they look a great deal like what expertise reversal predicts. I want to be careful with “predicts,” because the experts’ quality dip has other candidate explanations, overreliance and workflow disruption among them, and nothing in the economics papers separates those.
Expertise reversal describes what happens to performance when you hand someone support. Whether the novice ever becomes the expert is a different measurement.
What happened when the tool was taken away
When somebody made it, the answer was not encouraging. I wrote about this study a fortnight ago for a different reason, and it took me until now to see what it does to my own line. Nearly a thousand Turkish high school students got GPT-4 access for maths practice (Bastani et al., PNAS, 2025). While they had it, they did better. Then it was taken away for a test, and the group that had used the unrestricted version scored about 17 percent worse than classmates who never had access at all. A version built to give hints instead of answers largely removed that harm. The MIT essay-writing preprint from that same edition points the same way, with people who used the model first showing reduced neural engagement when they later worked unaided (Kosmyna et al., 2025), though that is a preprint with a very small swap session and I offer it as a weak signal rather than evidence.
The other side of this is genuinely strong. The Brynjolfsson team suggest their assistant works partly by spreading the practices of better agents to everyone else, and they raise the possibility that it produces real worker learning rather than borrowed performance. If that is what is happening, the novice is not being propped up, they are being taught by a system that has read what good looks like. Nobody can rule that out yet, because the support study measured output over months rather than capability after the tool was withdrawn.
There is also the obvious objection that an agent resolving tickets is not a student learning algebra, and that generalizing between them is exactly the sloppiness I complained about in the last edition. Fair. The Turkish study is doing the load-bearing work on learning here, and it is one study in one subject.
Where I have landed
So here is where I have landed, and I expect some of you to disagree.
AI reliably closes the performance gap. In the workplace productivity literature I have been able to read, I cannot find a study that measured whether it closes the capability gap, and the one careful classroom study that looked went the other way.
If that is right, then “AI levels the playing field” is a claim about output, and people have been hearing it as a claim about opportunity. Those are different promises, and the second one is what gets used to justify putting these tools in front of learners.
The size of that claim matters, so let me pin it down. The learning sciences have measured unaided post-test performance for decades and intelligent tutoring research does it routinely. My gap is narrow. It is that the large AI-at-work studies everyone is citing in this particular argument have not.
Which is why the answer here is better design. Keeping the tool away from the people with the least access to help would be its own inequity, and a worse one.
The instructional design people already have a name for the fix, and I did not know it when I started saying my line. It is fading, an idea that comes out of the scaffolding literature Wood, Bruner and Ross opened in 1976, in a paper that already worried about support creating dependence: you build the support in, then you deliberately withdraw it as competence grows. Bastani’s hints condition is fading, implemented in a chatbot. The design decision did the work. The technology was the same in both conditions.
I have changed one question rather than my position. When somebody asks whether to give a tool to people still learning the underlying skill, I ask what happens the first time it is unavailable, and whether anyone has planned for the support to come out on purpose. If nobody has thought about that, the tool is being deployed against performance and the learning is being assumed.
I am less certain about all of this than I was about the line I used to give that room, which is roughly the point.
How this was made. Full disclosure, and I mean it as a courtesy, not a confession. AI tools help me research, check my claims, tidy my grammar, and stay on top of far more reading than one person can. The thinking and the judgment are mine, I edit every word into my own voice by hand, and then I put my name on it. Telling you that is the whole point of keeping it human.
These editions run alongside Learning to Learn With AI, a longer series working the same ground in order.
Common questions
Does AI level the playing field for beginners? For output, the evidence says yes: novices gain the most while the tool is in hand. Whether that help turns into skill is a different measurement, and the one careful study of it found students did worse once the tool was gone.
Does AI help students learn? It depends on the design. In Bastani’s study, students with unrestricted GPT-4 scored about 17 percent worse once access was withdrawn, and a version that gave hints instead of answers largely removed that harm. Withdrawing support as skill grows, called fading, is the principle at work.
References
- Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192. https://www.science.org/doi/10.1126/science.adh2586
- Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at work. The Quarterly Journal of Economics, 140(2), 889-942. https://academic.oup.com/qje/article/140/2/889/7990658
- Dell’Acqua, F., McFowland, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organization Science. https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838
- Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89-100. https://acamh.onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.1976.tb00381.x
- Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23-31. https://www.tandfonline.com/doi/abs/10.1207/S15326985EP3801_4
- Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. PNAS. https://www.pnas.org/doi/10.1073/pnas.2422633122
- Kosmyna, N., Hauptmann, E., Yuan, Y. T., Situ, J., Liao, X.-H., Beresnitzky, A. V., Braunstein, I., & Maes, P. (2025). Your brain on ChatGPT: Accumulation of cognitive debt when using an AI assistant for essay writing task [Preprint]. arXiv:2506.08872. https://arxiv.org/abs/2506.08872