Detecting if ChatGPT or AI Wrote Your Content
With endless AI-based content writers in the market and the era of ChatGPT, it has become easy for anyone to write a blog.
ChatGPT can even help you to improve the grammar of your paragraphs or rephrase content from the internet. And as usual, people can easily pass both free and paid plagiarism checkers.
So, how do you know if you’re getting your money’s worth or if the content you’re consuming is valuable?
Era of ChatGPT
I’ve been using ChatGPT to communicate with the bot, understand difficult/new concepts, get precise information rather than choosing a link from Google, and write small pieces of content.
When you use ChatGPT a couple of times, you’ll realize that its language is machine-like and repetitive.
For example, when I asked it to write a summary of multiple movies, the structure went like this:
The “Movie Name” was released in 2017. It was directed by “Name of Director”. The movie follows “Actors and basic plot in one line.” Critics have been praised it for its sharp and witty script, direction, and performances, making it an instant classic in the horror genre. The film has an IMDb rating of “” and a Rotten Tomatoes score of “”.
I tried various prompts such as:
- Write the summary/plot/synopsis of
- Write the summary/plot/synopsis of + with plot twist
- What was X movie about
And yet, every time, the structure of the response was the same.
Unnatural Grammar and Syntax
If you notice awkward or repetitive sentence structures and vocabulary choices, the content is possibly generated by a machine rather than a human. It’s because most AI systems can not yet reach the level of human-like natural language writing.
When you interact with ChatGPT and you mention that the information is wrong, ChatGPT will apologize and tell you why its response is wrong.
Read about How ChatGPT Failed as a Research Tool.
Tools for Detecting AI Written Content
So, I obviously started by asking ChatGPT to tell me the tools to detect AI Written Content:
From the four responses, two websites: GPT Detection and AI Language Detection don’t exist.
Whereas, Copy.AI is one of the best AI-based copywriting tools and LanguageTool is a grammar and style checker tool and neither of them currently offers any service for detecting AI-based content.
Let’s come back to the tools for detecting AI-written content:
1. GPT-2 Output Detector Demo
GPT-2 Output Detector is an online demo of the GPT-2 output detector model, which is based on the 🤗/Transformers implementation of RoBERTa. After entering the text, the platform will display the Fake/Real probability of the text.
Review: I tested the content I have written for my research proposal and Medium, and it was 99.8% Real both times. However, when I tried content written for news and blogs, GPT-2 Output Detector labeled it 99.8% Fake, even when it was written by me. It was unclear of what factors are taken into consideration when labelling the content.
2. Content at Scale
Content at Scale’s AI Detector is an AI-based detection system that can spot if an article was written by a human or by GPT-3 technology. It uses natural language processing (NLP) and semantic analysis algorithms to identify AI-generated content. It also helps to optimize content for higher rankings on search engines and ensures that the content will pass the AI content detector.
Review: The above paragraph was written by Chatsonic, another AI-based chatbot based on ChatGPT with access to latest Google results. However, when I put it on Content at Scale and it results in 47% Real and 53% Fake. Does this mean that Chatsonic was able to write like humans or that the platform couldn’t detect it.
3. Originality.AI
Originality.AI is a paid tool that offers advanced tools such as a similarity checker, content comparison, and AI-powered rewriting. Since it’s a paid tool, I couldn’t test it. Here is what they have to say about ChatGPT and the accuracy of their tool:
With the release of OpenAI’s new model for AI-generative text, we needed to re-check Originality.AI’s accuracy. Essentially we completed a study to identify if the AI developed at Originality.AI can detect if the content that was produced by ChatGPT, GPT 3.5 (DaVinci-003) with the same accuracy as it can for GPT-3 which is 94%.
4. Writer
Writer is another paid tool for detecting AI-generated content, which you can try for free. I tested the content generated by Chatsonic for Content at Scale platform to test the originality.
Review: According to Writer, only 5% of the text was human generated. The platform suggested me to edit it, so it becomes more human like. Unlike Content at Scale that said the content was almost 50–50 Fake and Real, Write was better at detecting the AI-generated content.
Final Words
It was great testing AI content detectors because similar to plagiarism checkers, they aren’t built the same. Just like paid plagiarism checkers, Writer did an impressive job at detecting that content was written by AI, which was accurate.
If you haven’t tried out ChatGPT or Chatsonic yet, you definitely should, this is the future!
I appreciate you and the time you took out of your day to read this! You can find more articles like this from me on QuickGuide blog. For news and insights, please find me on Twitter at @osheenjain, to see what I do when I’m not working, follow me on Instagram.
Don’t forget to like and subscribe on Medium, this motivates me a lot!
