How Accurate Are AI Detectors?

Two kinds of errors

• A false positive is human writing labeled as AI. This is the error that hurts innocent people.
• A false negative is AI writing labeled as human. This is the error that lets misuse through.
• Every detector trades one against the other. Tuning a detector to catch more AI text usually flags more human writers too.

What detector makers have said

When OpenAI released its AI text classifier in early 2023, it reported that the tool correctly identified only 26% of AI-written text as likely AI-written, while wrongly labeling human writing as AI-written 9% of the time. OpenAI withdrew the tool on July 20, 2023, citing its low rate of accuracy. Today’s commercial detectors claim much better numbers, usually measured on their own test sets. Independent testing often finds higher error rates, especially on edited or unusual text.

What newer independent research shows

Detectors have improved. A 2025 study by University of Chicago economists Brian Jabarian and Alex Imas tested Pangram, Originality.ai, and GPTZero on 1,992 human-written passages and matching AI-written ones across six genres. With well-chosen settings, the best commercial detectors had very low error rates on unedited text, and Pangram’s false positive rate was near zero on medium and long passages. Results differed sharply once AI text was run through a “humanizer” tool: Pangram held up, while GPTZero missed a large share of the rewritten AI text. The open-source baseline they tested performed far worse than all three. The takeaway is that the best tools are much better than early detectors, but quality varies widely between products, and none of them can see how a text was actually written.

Why a “1% false positive rate” is not small

Vanderbilt University did this math in 2023 when it turned off Turnitin’s AI detector. Turnitin had stated a 1% false positive rate. Vanderbilt noted that it received about 75,000 paper submissions in 2022, so a 1% rate could have meant roughly 750 papers incorrectly flagged as AI-written. Behind each of those is a student who did the work and then had to defend it.

When accuracy drops

• Short texts. Under about 150 words there is too little to measure.
• Edited AI text. Rewording, mixing human and AI sentences, or running text through a paraphrasing tool lowers detection rates sharply.
• Non-native English writing. Research has found detectors flag it far more often than writing by native speakers.
• Formal and formulaic genres. Legal writing, lab reports, and five-paragraph essays follow predictable patterns.
• Newer AI models. Detectors trained on older models can lag behind new ones.

Some vendors build in caution

Turnitin, for example, does not show a number for AI scores between 1% and 19%. It shows an asterisk instead, because it says false positives are more common in that range, and it requires at least 300 words of prose before it produces a report at all.

Different detectors, different answers

Run the same text through several detectors and you will often get very different scores. That is not a bug in one tool; it reflects that each method measures different patterns and was trained on different data. Disagreement between detectors is itself a sign that a result should not be treated as conclusive.

How to read a score responsibly

• Treat it as a reason to look closer, not as a conclusion.
• Look at which sentences were flagged. A high score driven by a quotation or a list means little.
• Compare with the writer’s other work and talk with them about how they wrote it.
• Ask for drafts, notes, or version history before reaching any decision.
• Never base a grade, firing, or accusation on a detector score alone.