Newsletter AI Learning, Keeping It Human Call 647-641-0643
Christopher Ross
,

750 papers would have been wrongly flagged as AI-written

Christopher Ross

9 min read
A coil of thick, rusted chain lies partially unspooled, suggesting both connection and potential entanglement.
In this post

In 2022, Vanderbilt ran about 75,000 student papers through Turnitin. When Turnitin launched its Artificial intelligence (AI) writing detector the next year, it claimed a false-positive rate of roughly 1 percent.

Vanderbilt did the multiplication in public. At that rate, on that volume, around 750 papers would have been wrongly flagged as AI-written. They switched the detector off and said why (Vanderbilt, Brightspace, 2023).

Arithmetic graphic: 75,000 student papers multiplied by a claimed false-positive rate of about 1 percent equals about 750 papers wrongly flagged as AI-written, the calculation Vanderbilt made before switching the detector off.

One percent sounds like a rounding error until somebody multiplies it by a year of your submissions and reads you the total out loud.

Three years on, that decision stopped being unusual. Yale, Johns Hopkins and Indiana have restricted or banned detection outright, and Northwestern, Georgetown and New York University are among at least a dozen institutions that have switched off Turnitin’s AI detection (Inside Higher Ed, August 2026). Indiana’s Kelley School told its own faculty that tools claiming to detect AI use “are highly unreliable.”

Key takeaways

  • Vanderbilt switched off Turnitin’s AI detector: about 75,000 papers a year at a claimed 1 percent false-positive rate meant roughly 750 students wrongly flagged.
  • The errors weren’t evenly spread. A Stanford team found detectors wrong on more than half of essays by non-native English writers.
  • Detection taught students to hide their AI use, and every skill worth teaching about these tools needs that use to be visible.

The cost was never spread evenly

This diagram compares the performance of AI detectors on native and non-native English writers, showing the false positive rates and the impact of linguistic variety on detection errors. The detectors were found to be close to perfect on essays by US eighth-graders, but misclassified more than half of TOEFL essays written by non-native English speakers. Enriching the vocabulary of the essays reduced the misclassifications.
I map the false positive rates of AI detectors on different groups of writers.

Here’s the finding I’d put in front of any teacher with English-language learners in their room. Weixin Liang and colleagues at Stanford ran detector after detector over two sets of essays. On essays by US eighth-graders, the detectors were close to perfect. On TOEFL essays written by non-native English speakers, more than half were misclassified as AI-generated (Liang et al., Patterns, 2023).

The mechanism is the part that tells you who in your room is exposed. The detectors keyed on linguistic variety. A narrower working vocabulary and simpler constructions read as machine-like, which is a fair description of careful writing by somebody working in their second language. When the researchers enriched the vocabulary of those same essays, the misclassifications dropped.

Which means the student most likely to be flagged in your room is the newcomer who has worked hardest to write plainly and correctly.

The lesson it taught

That’s one harm, and it falls on students who did nothing. There’s a second, and it falls on everybody. Less gets written about what the machinery taught the room, and that’s the part switching the detectors off doesn’t fix.

Admitting you used AI carries a price, and it has been measured. Jessica Reif, Richard Larrick and Jack Soll at Duke ran a series of experiments on how people judge colleagues who use AI. Evaluators rated them less competent and less motivated than people who produced identical work without it, and the people using AI anticipated exactly that judgment before anyone made it (Reif, Larrick and Soll, PNAS, 2025). The same penalty showed up in assessments of job candidates.

The anticipation is the part that matters in a classroom, and I’m about to carry it somewhere Reif didn’t. They studied professionals, not teenagers. But I’d put money on this: your students aren’t waiting to find out whether disclosure costs them something. Most of them walked in having already decided that it does.

Before I put those two findings together, they are not about the same people. Liang’s detectors were misfiring on one specific group. Reif’s evaluators were colleagues judging professional competence, not teachers marking essays, and no study I can find has put a detector and a disclosure penalty in the same room to watch them interact.

Put them together anyway and I think you get this. The incentive to hide was already sitting there, and a detector adds enforcement to it. Hiding also works, which is the part worth being honest about. Yale’s people, in the same reporting, described the whole business as a cat-and-mouse game where the methods of evasion improve about as fast as the detection does. So the rational response to enforcement isn’t to stop using the tool. It’s to stop admitting to it.

You cannot teach judgment about a tool that nobody in the room will admit to using.

That’s what makes this bigger than cheating. Every skill worth teaching about these tools requires the use to be visible. Spotting when the output is wrong is one. Noticing that a tool has quietly done the thinking you were supposed to do is the harder one, and neither can be practised in secret. Concealment doesn’t just hide misconduct. It hides the practice you were going to teach.

Meanwhile, the guidance is thin

If you’re waiting for your state, your province or your board to resolve this, the waiting is the problem.

Chalkbeat reviewed the AI guidance issued by roughly 37 states and the District of Columbia. Academic integrity typically gets a page, sometimes a few paragraphs, and the documents largely skirt the question of what a teacher should actually do (Chalkbeat, September 2026). States are encouraging adoption faster than they’re answering the question adoption creates.

A school library director quoted in that reporting named the one thing still under a teacher’s control: the assignments themselves. Smaller lever than a policy, and a much faster one.

What that looks like on Monday

Four things, in the order I’d try them.

  • Say what you use. The fastest way to end a concealment norm is for the adult in the room to go first, specifically, without apologising for it. There’s a version of this at the foot of every edition of this newsletter. I’ve rewritten it a few times and I reread it before I publish, because asking you to do this while ducking it myself would be a poor way to spend your Thursday.
  • Make disclosure cost nothing. A one-line “what did you use, and for what” box, with an explicit promise that an honest answer is never itself a penalty. The Duke work is about adults at work, and I’d be surprised if students were the exception: they arrive expecting to be judged for admitting it. Whether saying the opposite out loud actually shifts that isn’t something I can find anybody having tested, so hold it loosely. What I’m confident of is that the norm gets set by what you do the first time somebody tests it.
  • Ask for the process, not just the artifact. A rough first attempt, or two minutes explaining one decision they made and why. Process evidence is hard to fabricate convincingly, and it’s what you actually wanted to assess.
  • Put one unaided attempt somewhere early. It gives you a baseline of what this student can do by themselves, which is what lets you read their later work properly. Tell them that’s what it’s for.

Three things you don’t have to build yourself, all free and usable this year. The TeachAI Guidance for Schools Toolkit has policy language you can adapt, which beats starting from a blank page. Common Sense Education’s AI literacy lessons for grades 6 to 12 are classroom-ready as they stand. And Stanford’s CRAFT collection is built for high-school teachers who want their students critiquing these systems, not just operating them.

What all of that builds is a room where you can see the work being done, and that’s the condition everything else depends on.

The schools switching off their detectors have worked out the first half of this. The second half, building the disclosure norm that has to replace it, isn’t a procurement decision, and nobody’s going to issue it as guidance. It gets built one classroom at a time, by teachers, starting in about week three.

One question I’d like answered, because I haven’t solved it. If you’ve built a disclosure norm your students actually trust, what did you do the first time one of them told you something awkward? I’ve never seen anyone write that part down.

Next edition: what to measure instead of satisfaction. If the only evidence anyone has ever put in front of you is an end-of-module happy sheet, agreeing on something better with your stakeholders is the whole fight. Held over from last week.

These editions run alongside Learning to Learn With AI, a longer series working the same ground in order.

How this was made. Full disclosure, and I mean it as a courtesy, not a confession. AI tools help me research, check my claims, tidy my grammar, and stay on top of far more reading than one person can. The thinking and the judgment are mine, I edit every word into my own voice by hand, and then I put my name on it. Telling you that is the whole point of keeping it human.

Common questions

Are AI detectors accurate? Not reliably enough to accuse a student. Turnitin claimed a false-positive rate of about 1 percent, which at Vanderbilt’s volume meant roughly 750 papers wrongly flagged, and a Stanford team found detectors wrong on more than half of essays by non-native English writers.

What is Turnitin’s AI detection false positive rate? When Turnitin launched its AI writing detector, it claimed a false-positive rate of roughly 1 percent. Vanderbilt had run about 75,000 student papers through Turnitin in a year, so that rate meant around 750 students wrongly flagged.

Why are universities turning off AI detection? Because of false positives that fall hardest on non-native English writers, and because detection pushes students to hide their AI use. Vanderbilt was among the first to switch Turnitin’s detector off, and at least a dozen institutions have followed.

References

Working through something on your own site? Get in touch →