Are AI Detectors Accurate? What the Research Shows

Are AI Detectors Accurate? What the Research Shows cover image

Quick answer: Not reliably enough to prove anything. AI detectors can often spot unedited ChatGPT text, but they also flag human writing as AI (false positives) and miss AI text that has been edited (false negatives). OpenAI withdrew its own AI text classifier in 2023 because of its low accuracy, and a Stanford study found detectors flagged most essays by non-native English writers as AI. Treat a detector score as one clue, never as proof.

Key takeaways

  • A detector score is a probability, not evidence of who wrote a text.
  • False positives are most common with formal, simple or template-like writing and with non-native English writers.
  • Light editing or paraphrasing can make AI text pass as human.
  • Short texts are especially unreliable.
  • If you are falsely accused, your drafts and version history are your best evidence.

How accurate are AI detectors, really?

AI detectors look for patterns that are common in machine-written text: predictable word choices, very even sentence lengths and a smooth, generic style. Raw output from a chatbot often has those patterns, so detectors catch a lot of it. The trouble is that plenty of human writing has them too, and AI text stops having them once someone edits it.

So accuracy depends heavily on the text. A detector can look excellent on unedited AI essays and still be wrong about a careful student, a technical writer or someone writing in their second language.

What the research shows

SourceWhat it found
OpenAI, AI text classifier (2023)On OpenAI’s own test set, the classifier caught 26% of AI-written text and wrongly labeled human text as AI 9% of the time. OpenAI withdrew it on July 20, 2023, citing its low rate of accuracy.
Liang et al., Stanford (2023)Seven popular detectors were near-perfect on essays by US 8th graders, but misclassified more than half of 91 TOEFL essays by non-native English writers as AI (average false positive rate 61.22%). 89 of the 91 essays were flagged by at least one detector.
Vanderbilt University (2023)Turned off Turnitin’s AI detector. It noted that even the 1% false positive rate Turnitin claimed at launch could have meant around 750 of the 75,000 papers Vanderbilt submitted in 2022 being wrongly flagged.

These studies are from 2023, and detectors have changed since then. The underlying problem has not: a detector can only estimate how much a text looks like AI writing, and some human writing simply looks that way.

Why human writing gets flagged as AI

  • Formal, impersonal style. Textbook-like writing with no personal voice reads like AI to a detector.
  • Simple, predictable wording. This is why non-native English writers, who often use more common words, are flagged more often.
  • Templates. Cover letters, lab reports and product descriptions follow fixed patterns.
  • Grammar and rewriting tools. Heavy polishing can smooth out the quirks that make writing look human.
  • Short texts. A paragraph or two gives the detector very little to go on.

Why AI writing gets past detectors

False negatives happen too. When someone edits AI output, adds their own examples, changes the structure or runs it through a paraphrasing tool, the statistical patterns weaken and the score drops. That means a low score does not prove a human wrote the text either.

What an AI detector score really means

Most detectors show a percentage. It is the tool’s confidence that the text resembles AI writing, not the share of the text that was written by AI, and not a measure of honesty.

ScoreReasonable reading
LowFew AI-like patterns. Not proof that no AI was used.
MixedSome sections look AI-like. Common for edited AI text and for polished human writing.
HighStrong AI-like patterns. A reason to look closer and ask questions, not a verdict.

❌ The detector says 90% AI, so the student used ChatGPT.
✅ The detector shows strong AI-like patterns, so I will look at the drafts and talk with the student before deciding anything.

What to do if you are falsely accused of using AI

  1. Stay calm and ask for specifics. Ask which tool was used, what score it gave and which policy applies.
  2. Show your version history. In Google Docs, go to File > Version history > See version history. In Microsoft Word with OneDrive, use File > Info > Version History. Timestamps of your drafts are strong evidence.
  3. Bring your process. Notes, outlines, sources you read, earlier drafts and browser history all show how the work came together.
  4. Offer to explain your work. Talking through your argument, or writing a short passage on the spot, shows you understand it.
  5. Point to the research. The OpenAI and Stanford findings above show that detectors make mistakes, especially with formal writing and non-native writers.

For teachers: using AI detectors responsibly

  • Never use a detector score as the only evidence.
  • Run the same text through more than one tool. Detectors often disagree.
  • Compare the work with the student’s earlier writing and ask about their process.
  • Be extra careful with English learners and with very short assignments.
  • Make your AI policy clear before the assignment, including which uses are allowed.

How to check your own writing

If you want to see how your text might be read, paste it into our free AI detector. It highlights the sentences that read most like AI, so you can see which parts look generic. Overused words are a common cause; our list of AI words to avoid shows plain alternatives. The honest fix is to add your own voice: specific examples, opinions and details only you could write.

Our own detector is no exception to the limits above. As we explain on its page, it is much less reliable on formal, encyclopedia-style human writing, so read its score as one signal.

Related tools and guides

Frequently asked questions

Can AI detectors be wrong?

Yes. They produce both false positives (human writing flagged as AI) and false negatives (AI writing that passes as human). OpenAI withdrew its own classifier in 2023 because of low accuracy.

Why does my writing get flagged as AI?

Formal, predictable or template-like writing looks statistically similar to AI text. Short texts and writing by non-native English speakers are flagged more often.

Can a teacher prove I used AI with a detector?

A detector score alone is not proof. It should be weighed with your drafts, version history and how well you can explain your work.

Which AI detector is the most accurate?

Accuracy changes with each update and depends on the text, so no detector is reliable enough to be used as proof. If you use one, check the same text with more than one tool.

Do AI detectors work on short texts?

Poorly. OpenAI noted its classifier was very unreliable below 1,000 characters, and most detectors need several paragraphs to give a meaningful score.

Sources

Check your writing in seconds

Paste your text into the free SpellMistake grammar checker to catch spelling, grammar and punctuation mistakes instantly.

← Back to the blog