28.12.2025 • 7m Read Time • Authors: Noah Hafner
Which AI is best at moderating inappropriate content?
Introduction
Since the inception of the web people have always been misusing platforms to spread malice, spam, and other forms of bothersome content. Despite being the 21st century it's shocking to see that some platforms are still suffering from extreme spam and misuse, companies typically have to employ thousands of human "content moderators" in LEDCs to keep their platforms "clean". Not only is this immoral1,2,3 but it's blatantly ineffective4,5.
As part of an investigation I've opted to put some of our top LLM's head-to-head to identify which LLM performs the best at content moderation.
P.S: At some point I'm planning on conducting an experiment to see if people can bypass AI moderation and then from that collect a larger pool of results (which in turn should be more accurate).
The Benchmark
MischiefBench works by challenging a variety of LLM's with the same scenarios and policies and then analysing which LLM makes the correct decisions. Each LLM can expect a scenario to look a little bit like this:
{
"id": "gaming_001", // ID of the scenario
"context": "video game chat", // The context of the chat.
"content": "I will kill him", // The content of the "message".
"target_age": 13, // The minimum age that the LLM should expect to see the content.
"policy_ref": "gaming", // Reference to the policy.
"expected": true, // Is this content acceptable? (true and hence acceptable.)
"difficulty": "medium", // How much we expect the LLM to struggle with this challenge.
"notes": "In-game violence reference, contextually appropriate" // Internal notes.
},
...
I thought during the benchmark it would be crucial that each LLM receives context regarding the specific scenario, this is for the reason that "I will kill you" may be acceptable in the sense of a First Person Shooter, but maybe not so much on a messaging app... Throughout the benchmark the LLM can expect to handle up to 235 scenarios in the following contexts:
I. Dating App
II. Educational App (Exchange for students & teachers)
III. Web Forum
IV. Gaming Scenarios (E.G: In-game chats)
V. Health & Fitness App
VI. Marketplace App
VII. Professional Exchange App
VIII. Social Media App
IX. Streaming App
It's important to note that the LLM has different policies that it must adhere to for each different category, the following is an example of part of the policy for the educational app:
{
"name": "educational", // Name of the policy.
"context_description": "Educational platform for students and tea...", // Description of the policy.
"min_age": 10, // Expected minimum age to witness on the platform.
"rules": [
{
"category": "appropriate_content", // Category of rule.
"description": "All content must be age-appropriate and educational in nature", // Description of rule.
"severity": "high", // Severity of the rule.
"examples": [ // Examples of violating content.
"Acceptable: Homework help, study materials, educational discussions",
"Unacceptable: Adult content, violence, inappropriate jokes"
]
},
{
"category": "school_location",
"description": "Sharing specific school names or locations poses safety risks",
"severity": "high",
"examples": [
"Unacceptable: 'I go to Lincoln Elementary', sharing school address"
]
},
...
All LLM's were queried with the exact same system prompt and all requests were filed via openrouter.
The results
In terms of accuracy the following results could be concluded. Prefer a graphical representation of the results? Scroll down!
Ranking by Accuracy
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Rank ┃ Model ┃ Accuracy ┃ Correct/Total ┃ Avg Latency ┃ Total Cost (235 tests) ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ #1 │ Claude 3.5 Haiku │ 94.89% │ 223/235 │ 2160ms │ $0.1032 │
│ #2 │ Gemini Flash 3 Preview │ 94.04% │ 221/235 │ 1064ms │ $0.0561 │
│ #3 │ Llama 3.3 70B │ 94.04% │ 221/235 │ 1679ms │ $0.0186 │
│ #4 │ GPT-4o │ 93.62% │ 220/235 │ 756ms │ $0.2133 │
│ #5 │ Gemini Flash 2.5 │ 93.62% │ 220/235 │ 528ms │ $0.0425 │
│ #6 │ OpenAI: gpt-oss-120b │ 92.77% │ 218/235 │ 2563ms │ $0.0086 │
│ #7 │ DeepSeek: DeepSeek V3.2 │ 92.34% │ 217/235 │ 3248ms │ $0.0199 │
│ #8 │ Claude 3.5 Sonnet │ 91.91% │ 216/235 │ 2351ms │ $0.7613 │
│ #9 │ xAI: Grok 4.1 Fast │ 91.49% │ 215/235 │ 3466ms │ $0.0493 │
│ #10 │ GPT-4o Mini │ 91.06% │ 214/235 │ 1140ms │ $0.0133 │
└────────┴─────────────────────────┴──────────┴───────────────┴─────────────┴────────────────────────┘
Ranking by Speed
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Rank ┃ Model ┃ Avg Latency ┃ Accuracy ┃ Total Cost ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩
│ #1 │ Gemini Flash 2.5 │ 528ms │ 93.62% │ $0.0425 │
│ #2 │ GPT-4o │ 756ms │ 93.62% │ $0.2133 │
│ #3 │ Gemini Flash 3 Preview │ 1064ms │ 94.04% │ $0.0561 │
│ #4 │ GPT-4o Mini │ 1140ms │ 91.06% │ $0.0133 │
│ #5 │ Llama 3.3 70B │ 1679ms │ 94.04% │ $0.0186 │
│ #6 │ Claude 3.5 Haiku │ 2160ms │ 94.89% │ $0.1032 │
│ #7 │ Claude 3.5 Sonnet │ 2351ms │ 91.91% │ $0.7613 │
│ #8 │ OpenAI: gpt-oss-120b │ 2563ms │ 92.77% │ $0.0086 │
│ #9 │ DeepSeek: DeepSeek V3.2 │ 3248ms │ 92.34% │ $0.0199 │
│ #10 │ xAI: Grok 4.1 Fast │ 3466ms │ 91.49% │ $0.0493 │
└────────┴─────────────────────────┴─────────────┴──────────┴────────────┘
Ranking by Cost Efficiency
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Rank ┃ Model ┃ Efficiency Score ┃ Accuracy ┃ Cost/Scenario ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ #1 │ OpenAI: gpt-oss-120b │ 10757.39 │ 92.77% │ $0.000037 │
│ #2 │ GPT-4o Mini │ 6861.89 │ 91.06% │ $0.000056 │
│ #3 │ Llama 3.3 70B │ 5059.86 │ 94.04% │ $0.000079 │
│ #4 │ DeepSeek: DeepSeek V3.2 │ 4637.22 │ 92.34% │ $0.000085 │
│ #5 │ Gemini Flash 2.5 │ 2201.26 │ 93.62% │ $0.000181 │
│ #6 │ xAI: Grok 4.1 Fast │ 1854.53 │ 91.49% │ $0.000210 │
│ #7 │ Gemini Flash 3 Preview │ 1676.64 │ 94.04% │ $0.000239 │
│ #8 │ Claude 3.5 Haiku │ 919.67 │ 94.89% │ $0.000439 │
│ #9 │ GPT-4o │ 438.83 │ 93.62% │ $0.000908 │
│ #10 │ Claude 3.5 Sonnet │ 120.73 │ 91.91% │ $0.003240 │
└────────┴─────────────────────────┴──────────────────┴──────────┴───────────────┘
Conclusion
AI moderation may be an effective way forward for moderation of the web, it's more ethical than human moderators, it's cheaper, it's quicker and it has good scalability. I solemnly believe that AI should play a crucial role in moderation of the web in the future. Thanks for reading.
References
1. Arsht, A. and Etcovitch, D. (2018) The human cost of online content moderation, Harvard Journal of Law & Technology. Available at: https://jolt.law.harvard.edu/digest/the-human-cost-of-online-content-moderation (Accessed: 28 December 2025).2. Lauren, W. and Deepa, S. (2017) The Worst Job in Technology: Staring at Human Depravity to Keep It Off Facebook. Available at: https://www.wsj.com/articles/the-worst-job-in-technology-staring-at-human-depravity-to-keep-it-off-facebook-1514398398 (Accessed: 28 December 2025).
3. Spence, R., & DeMarco, J. (2025). Content Moderator Mental Health and Associations with Coping Styles: Replication and Extension of Previous Studies. Available at: https://doi.org/10.3390/bs15040487 (Accessed: 28 December 2025).
4. Gongane, V. U., Munot, M. V., & Anuse, A. D. (2022). Detection and moderation of detrimental content on social media platforms: current status and future direction. Available at: https://doi.org/10.1007/s13278-022-00951-30 (Accessed: 28 December 2025).
5. Amrita V (2023). It's Time to Rethink the Standard Frame of Content Moderation. Available at: https://www.cigionline.org/articles/its-time-to-rethink-the-standard-frame-of-content-moderation/ (Accessed: 28 December 2025).