Do AI writing tells actually work? I tested 15

September 4, 2026 · every figure below recomputes from two files in the repo

Every list of AI writing tells looks about the same. Em dashes. The word delve. Sentences shaped like "not just X, but Y." The lists circulate, grow a few entries each month, and end up quoted in arguments about whether a student wrote her own essay. Very few of them carry a number.

I wrote one of those lists. Fifteen rules, shipped as the free checker on this site, built the way most of them get built. I noticed things, and I wrote regular expressions for them.

Then I went looking for a way to check whether the rules were right.

What the test needed

Writing a model could not have produced. Not writing that looks human, writing that is provably human, and the only proof that survives an argument is a date. ChatGPT opened to the public on November 30, 2022. Anything published and indexed before that day was typed by a person.

So the human side of this study is 959 documents and 1,354,970 words, every one dated before that cutoff: Substack posts indexed in their own publications' 2021 sitemaps, blog posts from the 2004 Blog Authorship Corpus, Medium articles, and email newsletters. Against them sit 194 documents generated this year by Claude, GPT-5.5, ChatGPT, and Gemini, in matched genres.

Every rule gets a rate per thousand words and a confidence interval built by resampling documents, because one long article is one observation and not four hundred sentences. A rule counts as working when its interval clears the human interval with no overlap.

Em dashes are a Claude fingerprint

Raw em dashes per thousand words, same measurement across every corpus:

GPT-5.5 uses fewer em dashes than Medium writers did. So does Gemini. If a heavy em-dash habit makes you suspicious, you are detecting Claude, and you are also detecting about one Medium article in seven.

The other fourteen

None of them separated. Not on a corpus of 959 human documents, and not against any of the four model families. The closest miss was contrastive negation, where the intervals overlapped by 0.021 per thousand words, which is close enough that the measurement script now prints that margin instead of a bare yes or no.

Then I ran the checker over the human documents themselves. 602 of the 959 came back with at least one flag. That is 62.8% of writing produced before ChatGPT existed.

One more, because it is the second most-cited tell after the em dash. People say a flat, even sentence rhythm gives a model away. Measured as the variation in sentence length within each document, Claude runs 0.700 and the 2004 bloggers run 0.697, with Medium at 0.659 and Substack at 0.684. The most variable corpus in the study is the machine.

What this does and does not show

AI writing may well have a signature. What these numbers establish is that fifteen attempts to define one mostly failed, which is a smaller and more boring claim, and the difference matters. Absence of evidence is not evidence of absence. A better rule may exist, and I am still looking for one.

The useful part is narrower. A rule that fires on two thirds of human writing tells you nothing about the document in front of you. If somebody has been accused on the strength of one, the rate is the answer.

Then why does this site still hand out a fix

Being flagged and being caught are different problems. Most of these patterns do not identify a machine. They will still get you accused, because readers believe they do, and a reader who believes it will not run a bootstrap over 959 documents before deciding what she thinks of your email.

The instruction files on this site are for that second problem. They steer a model away from patterns people react to, which is worth doing whether or not the patterns mean anything. What changed is that this site no longer treats the two problems as one.

What changed here

The checker still runs, and it still highlights all fifteen. What it no longer does is imply that a highlight is a verdict. Every flag now carries the rate behind it, so "your draft has four em dashes" reads as a fact about your draft rather than an accusation about its author.

If someone has told you that an em dash or a rule of three proves a machine wrote your work, both files are public and the study code is in the repo. Take the rates into the conversation.

Part of HumanSounding. The human baseline is 959 documents dated before November 30, 2022, and every rate on this page recomputes from the two linked files.