How reliable are AI writing detectors? Orelien novel sparks debate

Claims that Haitian-Canadian author Thelyson Orelien used AI to write his debut novel have reignited questions about the accuracy of AI detection tools

Stay Connected, Stay Informed - Follow News Alert on WhatsApp for Real-time Updates!

Claims that Canadian-Haitian writer Thelyson Orelien used artificial intelligence to write much of his best-selling debut novel have sparked controversy in France and renewed debate over how reliably AI-generated writing can be identified.

The controversy began after an X account claimed that AI detection software Pangram had determined that Orelien’s novel was largely written using a text generator such as ChatGPT or Claude.

Orelien, 38, has denied the allegation and said he intends to prove that he wrote the novel himself. The book, which tells the story of a Haitian migrant travelling to Canada, has reportedly been sold to publishers in more than 20 countries.

“I didn’t write this novel with my head, but with my guts and my heart,” Orelien told AFP.

AFP tested excerpts of the novel using several AI detection tools and received widely different results. Some systems rated the text as very likely to have been written by a human, while another assessment classified it as 100% AI-generated.

How AI writing detectors work

Dozens of AI detection tools are now available, particularly for schools, universities and publishing organisations.

These systems generally compare text against large collections of human-written and AI-generated material. They analyse patterns in vocabulary, sentence structure and other linguistic characteristics to estimate whether a passage was produced by an AI system.

Research into the technology has advanced rapidly. A University of Chicago paper published in October 2025, titled “Artificial Writing and Automated Detection”, reported that Pangram achieved near-zero error rates when analysing medium-to-long text extracts.

However, experts say accuracy claims should be treated cautiously, particularly when the tools are used to make definitive judgments about individual writers.

What clues do AI detectors look for?

Thierry Poibeau, a researcher at France’s CNRS specialising in natural language processing, said AI detection systems look for recurring linguistic patterns or “tics” associated with generated writing.

These can include repetitive sentence structures, predictable wording and certain stylistic habits. Poibeau cited triadic phrasing, in which ideas are presented in groups of three, and frequent use of em dashes as examples.

Pangram says it identifies AI-generated text by analysing subtle patterns in writing. Some frequently repeated words, phrases and sentence structures can also contribute to an AI score.

OpenAI says AI agents mistakenly posted user images online

Can AI detectors be trusted?

Despite increasingly sophisticated technology, experts caution that AI detectors are not necessarily definitive.

Many detection services publish accuracy percentages, but such figures can be difficult to independently verify and may not fully account for false positives, in which human-written material is incorrectly identified as AI-generated.

Pangram CEO and co-founder Max Spero told The Atlantic that the software should be treated as a starting point for further investigation rather than as the final authority on whether a text was written by AI.

Poibeau also described AI detection as a “moving target” because generative AI systems are continuously changing.

The problem may be even more complicated when literary works are involved. A detector could identify repetitive structures or stylistic patterns that are actually deliberate features of an author’s writing style.

Orelien has argued that his prose reflects Haitian and Caribbean literary traditions and told Liberation that the software was more accustomed to French literary traditions and classical authors.

Experts remain divided over Orelien’s novel

Benoit Raphael, an entrepreneur and writer specialising in AI, conducted his own tests of Orelien’s novel using Pangram and also received results indicating AI involvement.

He told Nouvel Obs that results approaching 90% or 100% AI-generated were significant indicators, although they do not by themselves establish how the text was produced.

Questions about the novel’s style had also emerged before the controversy. Lea Bory, speaking on her podcast Torchon, reportedly suggested earlier this month that the book may have been AI-generated because of repetitive sentences and extensive use of analogies.

Writing in Canard Enchaine, author Fabrice Colin similarly criticised what he described as formulaic sentence structures and said he struggled to find a distinctive voice in the prose.

The controversy has therefore highlighted a broader problem facing publishers, writers and readers: AI detection technology may provide useful clues, but determining whether a literary work was actually written by a human remains more complicated than a single percentage score might suggest.

Leave a Comment

This material may not be published, broadcast, rewritten, redistributed or derived from.
Unless otherwise stated, all content is copyrighted © 2025 News Alert.