Added Openai text classifier to the list of tests below.
Unless you have been hiding under a rock, you will have seen the rise in AI-written content, particularly the rise of ChatGPT by openAI.
Everyone is turning to ChatGPT and AI-written content to boost their writing output, but is that a good thing?
So far, I can say yes it is a good thing. However, we are starting to see some problems arising.
One problem I’m starting to see is nonsensical content. Content that reads okay but does not make sense. Another problem is for content buyers. You want to be sure the content is written by a human and is not plagiarized. And finally, you’ll not want to have the content penalized by Google.
For these reasons and some others, more AI detection tools being developed. In this article, I will discuss the popular AI detectors you can use to see if your content has been written by AI and if you may be in danger of a Google penalty.
My AI Detection Tool Tests
Before I get into the nuts and bolts of AI detectors i thought it would be a good idea to actually test them and record the results. I used ChatGPT to generate 2 paragraphs about double decker buses. I then used this text in all the tools tested so i could compare the results. So here we go.
Types of AI Detector Tools
There are many types of ai detectors using various technologies to sniff out AI. Too many to discuss here, and for this article, I am only talking about detectors of AI writing using language models such as GPT2, GPT-3, and ChatGPT.
For further simplification, I categorized the detectors into 3 groups, free, paid, and academic.
Free AI detection Tools
Who does not like free tools? Yes, they are great, although they do have their limitations too. However, some people only require the odd article check now and again. For these people, free tools are a good option.
Free Giant Language Model Test Room (GTLR)
So we do not get too technical here, I have summarized the information from the GLTR demo. Basically if the output is colored mostly green then its considered AI written. It’s limitation is up to GTP2
- The GLTR software uses machine learning to detect text generated by large language models like GPT-2.
- It is a collaborative effort between researchers from the MIT-IBM Watson AI lab and Harvard NLP.
- GLTR analyzes the likelihood of each word being predicted given the context to the left.
- Color coding of the text is used to indicate the likelihood of the word being predicted: green for the Top 10, yellow for the Top 100, red for the Top 1000, and violet for words outside of the Top 1000.
- The tool has histograms with statistics about the text. It includes the fraction of probability for the actual word and the entropy along the top 10 results for each word.
- This version of GLTR was made in 2019.
Despite being made to detect GPT2 the GLTR managed to detect my ChatGPT test content successfully.
You can test your own text on the Giant Language Model Test Room by following this link http://gltr.io/dist/index.html
Corrector App – Free AI Content Detector
Corrector App is a free AI detection tool that detects text written by AI. It uses advanced algorithms for the detection according to their website. The app is one of a suite of text tools used for improving the quality of your written work. In addition to the AI text detector, the corrector app offers an AI-powered plagiarism checker, which can identify previously used content in your writing with little effort. With Corrector App, you can stay on top of your writing skills and avoid making mistakes.
The results from the free detector on the corrector app was not good. It did not detect the test AI content generated by ChatGPT. the corrector app includes a plagiarism checker and other writing assistant tools – follow this link for more information https://corrector.app/ai-content-detector/
Free HuggingFace GPT-2 Detector
The HuggingFace GPT-2 Output Detector Demo is a free tool for detecting the output of the GPT-2 text generation model. It is designed to help users avoid the potential risks posed by using AI generated content. Once the text is input, the detector quickly identifies any possible GPT-2 output. This ensures users get an answer to whether a piece of content is AI-generated.
As you can see in the image above the HuggingFace detector did not detect our ChatGPT test content. Note that HF is made to detect GPT2 and our test content was generated with ChatGPT-3.5 To read more about HuggingFace follow this link. https://openai-openai-detector.hf.space/
Sapling.ai Free AI Content Detector
Sapling is an AI-powered toolsuite that helps customer-facing teams respond to messages more quickly by providing real-time suggestions for personalized responses. It works in conjunction with CRMs and messaging platforms. They have several AI tools for use with chat and support systems and improving language and grammar
The sapling AI detector successfully identified our ChatGPT sample text as AI. To learn more about Sapling click the following link. https://sapling.ai/utilities/ai-content-detector
Paid AI Detection Tools
The paid detection tools do tend to perform better than the free ones. The paid service you choose will depend on the amount of content you need to check and how frequently. If supplying written content or buying content having an AI detector in your tool kit is essential nowadays.
Writer.com – AI Content Detector
Writer.com – AI Content Detector is a tool that can detect plagiarism in content quickly and accurately according to their website! It uses algorithms to identify copied text from other sources with a probability higher than chance. This makes it a reliable tool for detecting AI or plagiarism in many types of written work. Additionally, it offers a variety of features to help students write content effectively and efficiently, such as real-time feedback on the writing process and detailed analysis of detected plagiarism.
I must say I was expecting better results from the writer.com detector tool. I don’t know why, maybe it was because it’s a name I’m familiar with. Anyway, the result was mixed and it suggested that most of the test text by ChatGPT was written by humans at 75% and 25% AI generated. Why not try it yourself, go here https://writer.com/ai-content-detector/
Originality.ai – AI Detector Built for Content Publishers
Originality.ai is a unique tool designed specifically for content publishers looking to protect their brand. It uses several algorithms to detect plagiarised work and AI-generated writing with precision.
The Originality.ai detection tool is easy to use and provides accurate results quickly, allowing you to identify ai generated and plagiarized content before it becomes a problem. The unique features of this tool make it a must-have for content creators, you can scan your whole site with one click.
The originality AI detection software performed excellently and detected the AI content with ease. I tested multiple pieces of AI generated text and it detected the AI content every time with 100% success rate with my tests. I recommend you use originality.ai for AI content detection
Copyleaks AI Content Detector
Copyleaks AI Content Detector is a powerful tool that can detect plagiarized content in multiple languages. It uses advanced artificial intelligence algorithms to scan texts for similarities and decides whether the content is plagiarized. This tool can be a helpful resource for students and professionals who want to ensure their work is original and not plagiarized. It offers a variety of features to make the detection process easier and more efficient, including automated citation generation and report generation.
Copyleaks AI Content Detector can detect plagiarism in long and short texts, such as essays, research papers, blog posts, and more. It is a reliable tool that can help individuals stay up-to-date with the latest research and trends in their field.
AI Content detector by copyleaks easily detected the AI sample text that I input. It didn’t give a numeric score just a probability message. They say the tool is still in beta so i assume they are continuing to refine and improve it. To find out more visit their site here: https://copyleaks.com/features/ai-content-detector
AI Detector Pro
AI Detector Pro employs a neural network to spot trends in AI text generation models like GPT-3. These are commonly used in online platforms for creating or revising AI content. Users can use the program to determine whether a text was created by humans or by artificial intelligence.
Users can paste text from a source or input the URL of a website to enter the text they want to check into AI Detector Pro. The program will then examine the text and assess the likelihood that an AI model produced it. Users will receive a report emphasizing the most suspect paragraphs, a probability score, and a paragraph-by-paragraph analysis.
The AI Detector Pro failed to identify the test content from ChatGPT, however it did detect the AI content in other tests I performed so you could say it was hit and miss. I’m trying to stay neutral with these tests but when a product is called “pro” and is a paid product, then it’s difficult not to be negative when it fails a simple test. As with all the products they are constantly being updated. To learn more about this product visit the website. https://aidetector.pro/
OpenAI Text Classifier For Indicating AI-written Text
OpenAI has developed a sophisticated text classification tool, which endeavors to differentiate between text written by human authors and AI-generated content from a variety of sources. This development looks to curb the rise of false claims that purport to present AI-generated text as being authored by humans. Examples of false claims include the use of automated misinformation tactics, the deployment of AI tools for academic cheating, and the masquerade of an AI chatbot as a human.
However, Openai say it is essential to acknowledge that the classifier is not infallible in its abilities. In OpenAI’s assessment of an English language “challenge set,” the classifier correctly identified 26% of AI-generated text as “likely AI-written,” yet it also made false detections of human-written text as AI-generated in 9% of instances. The reliability of the classifier tends to increase as the length of the input text increases. Despite its limitations, this latest classifier developed by OpenAI has demonstrated marked improvement over their previous iterations.
It appears that OpenAI are being cautious about declaring if text is human authored or AI generated. The result scores for any test are as either very unlikely, unlikely, unclear if it is, possibly, or likely AI-generated. There’s no percentage or numbers attributed and it all sounds a little vague. The result from our test was “unclear if it is Ai-generated” which falls in the middle of the 5 possible outcomes. I don’t know why but I was expecting something a little more positive from Openai. The AI text classifier is here https://platform.openai.com/ai-text-classifier
Academic AI Detection Tools
Academic AI detection tools work differently from the standard AI detector looking for AI writing in general content such as web pages. Academic AI detectors look for ideas in addition to written work, and the need for greater accuracy and no false positives is vital. The consequences of false positive detection of academic papers cannot be overstated. Let us look at a couple.
GPTZero AI Detection Trial
GPTZero AI Detection Trial is a tool for detecting ai written content. University student named Edward Tian created an app called GPTZero. It can detect whether a text was written by a human or by the artificial intelligence tool ChatGPT. GPTZero is a product aimed at educational establishments such as schools, colleges and universities.
GPTZero examines two variables in a piece of writing, including “perplexity” which measures randomness and “burstiness” which measures variance within a text. The app can help teachers detect if a text has been written by ChatGPT, which can prevent students from cheating. The beta version of the app is available for testing.
GPTZero performed well with the test text. As you can see from the image above it predicted the content was AI written with 96% probability. However it did fail some other tests I performed with it. With this being the beta demo and only released a few weeks ago I’ll give it more time to develop. There’s little information on their website but you can try the demo. https://gptzero.me/
Turnitin AI Writing and ChatGPT Detection
There was no test performed with the Turnitin ChatGPT detector. That is because the tool is not publicly available at the time of writing this in early January 2023.
I’ve included it here for information to show another example of AI detectors in educational settings.
The Turnitin artificial intelligence-based plagiarism detection tool uses a combination of advanced machine learning and human reviewers to identify plagiarism in written content. It can detect several forms of plagiarism.
Video Demonstration – Turnitin AI Detection
This demonstration shows how Turnitin AI Detection software works. It is a tool for educators who want to ensure that their students are not submitting genuine original papers that have not been plagiarized.
Video Link https://youtu.be/g85aB8qaSGc
AI Content Detection
Can AI writing be detected?
How to detect AI-generated content
For humans, it is nearly impossible to detect AI-generated content. It is easy for content producers to pass off AI-written content as human-written content.
Common signs of AI-generated content
Some say you can see patterns in AI writing, like repetition, short sentences, and factually wrong information. Personally, I could not detect AI written content, and I rely on AI detection technology.
I recommend you don’t waste time trying to detect AI-written content and use software detection instead.
These tools are essential for businesses looking to protect their reputation and keep their content within Google guidelines. They can help organizations detect plagiarism and other forms of content deception. However, companies must ensure they use the right detection tools for each type of content. Using GTP2 detection software to detect ChatGPT-3.5 content will not give accurate results.
Techniques for detecting AI-generated content
The best AI detection tools analyze content using several techniques, such as natural language processing, deep learning, and text analysis.
Some use probabilities by predicting the next word and then comparing the actual word. A high number of probable hits makes it more likely AI written. A new tool recently developed uses perplexity as a measurement with others.
Can Google detect AI-generated content?
Google has the technology to detect AI-generated content, but as it stands today, no one but Google knows if they are using it in the search algorithm as a ranking factor or penalty.
What we do know is that Google considers AI-generated content as spam and compares it to text spinning. Although I feel that is an odd comparison and categorization. I think this is Google being defensive until they have integrated AI detection. It does not make sense for Google to penalize just AI content alone. They will have to combine it with many other factors to avoid penalizing sites with false positive AI detection.
Can AI writing be detected?
The simple answer is yes. AI writing can be detected using one of the many AI detectors available. However, the accuracy of these tools varies depending on several factors. First, you should ensure that the detector you use is for the type of AI you want to detect. For example, don’t use GPT-2 detectors to identify GPT-3 content or ChatGPT-3.5.
Final thoughts and recommendations.
AI writers and tools like ChatGPT are helpful and provide many benefits to content producers, bloggers, and businesses. It’s how the tools are used that will make the difference.
If they are used in content factories to write large amounts of written copy for money, chances are the content will not be checked or edited for quality or accuracy.
Another area they can be misused is in translation. If the user is trying to write large quantities of content in a foreign language. It will be impossible to edit, check for factual accuracy or accuracy of the translation if the person using the AI tools cannot speak the language they are translating.
I feel AI content writers and ChatGPT are best used as an assistant or researcher, with the final written content being edited and written by humans. The human should check any facts offered and add their own voice and narrative to the content. Sometimes the AI may not save time but improve the content by providing more ideas or views from a different perspective.
Frequently Asked Question
Will Google penalize AI content?
Will Google Penalize AI content?
Google does not penalize content just for being AI-generated. However, they do consider it spam. Therefore content producers should familizes themselves with the Google definition of webspam.