I Tested 4 Different AI Image Detectors: Here's How Accurate They Were (2024)

If identifying fake art and photos is important to you, AI image detectors are invaluable—but some are more reliable than others. After testing the best apps available today, here’s how they each performed with AI-generated and human images.

How I Tested AI Image Detectors

After picking four of the best AI image detector tools around, I created two AI images on DALL-E 2. One is of a cat watching a butterfly. The other mimics a painting of a sailboat surrounded by birds.

Illuminarty, AI or Not, Hive Moderation, and SDXL Detector processed both of these images alongside a photo of elephant figurines definitely taken by a human.

The screenshots below display the initial results. Repeating the process didn’t change any outcome, which means that none of the apps learn as you use them. Their analysis and probability scores are based on their algorithms alone. You’ll find them listed from most to least accurate in detecting AI images.

How Accurate Was Hive Moderation?

Hive Moderation was the most effective AI image detector of the lot. The cat and butterfly image scored 99.9 percent. The sailboat was also marked as AI-generated with 98.3 percent certainty.

When given any image made by a human, Hive Moderation never gave a false positive. It can clearly tell the difference between real and artificial creations.

So, not only is Hive Moderation accurate, but it’s also very confident in its analytical capabilities. AI image generators struggling with hands stand little chance of fooling this AI detector.

How Accurate Was Hugging Face's SDXL Detector?

While not the most attractive app, the SDXL Detector on Hugging Face performed almost as well as Hive Moderation, making it the second most accurate AI image detector on this list.

It scores uploaded files based on their likelihood of being AI versus human. The cat and butterfly image got 0.844 for artificial and 0.156 for human. The detector accurately identified it as AI-generated but with some uncertainty.

The fake painting of a sailboat was correctly caught by Hugging Face’s algorithm. The artificial score was 0.981 and the human 0.019. The detector matched Hive Moderation’s probability calculation.

It was also right about the photograph of the elephant figurines. SDXL Detector marked it as human by a score of 1, leaving no chance of it being artificial.

How Accurate Was Illuminarty?

Illuminarty correctly spotted the AI images but thought the real photo was artificial, too. Let’s break down the results.

The cat and butterfly got the highest probability score: 89.3 percent. The detector is programmed to identify AI-generated images through their flaws, and it did quite well with this picture.

It was less confident, however, when it came to the AI artwork of a sailboat. The strokes and textures are convincing enough to give the detector pause but not fool it completely. It got a probability score of 78.5 percent.

But then Illuminarty was 77.7 percent sure that the photograph of the elephants was AI-generated.

This goes to show that even the best AI image detectors aren’t perfect. They can make mistakes depending on a picture's subject and characteristics.

How Accurate Was AI or Not?

AI or Not offers much simpler results. There are no probability percentages, just the likelihood of an image being AI or human.

When tested with the AI images, it correctly classified the cat and butterfly as artificial. The sailboat, however, tricked the detector and passed as human-generated.

Unfortunately, AI or Not struggles to process real photos and artworks, but when it does, it accurately identifies them as human-made.

What this experiment demonstrates is that some AI image detectors are still works in progress, while others are already extremely reliable in spotting fake pictures. While Hive Moderation and Hugging Face’s SDXL Detector are the most accurate, don’t discount the handiness of AI or Not as its software develops.

The best way to spot and avoid AI-generated images, especially as they become more and more realistic, is to have multiple detectors analyze them. Pick at least two that you trust, and don’t fall for a fake picture again.