OpenAI's new image classifier is designed to identify visual signs of child sexual abuse material (CSAM) with a specific signal for AI-generated content, a first for a broadly available tool of this type. By providing scores to help human reviewers prioritize high-risk images, the tool aims to streamline the detection of novel and previously unseen abusive material.
Improved Detection and Review Efficiency
The new image classifier was developed with assistance from the National Center for Missing and Exploited Children (NCMEC) to detect CSAM across various formats. It is specifically designed to identify visual signs of abuse in both traditional and AI-generated imagery, returning scores that allow safety teams to prioritize their review queues.
Chart drawn by HN Digest from figures reported by openai.com; values are shown as written in the source.
Early evaluations indicate significant improvements in efficiency. In tests comparing the new tool to existing protections, the new classifier flagged 48% fewer images while maintaining a similar detection rate for abusive content. Furthermore, images flagged by this new system were approximately twice as likely to contain actual abuse.
“The new classifier could reduce review work: we estimated that it would flag 48% fewer images while finding roughly the same number of abusive images as the existing tool.”
— Strengthening the fight against online child sexual abuse with new tools and support
Product Deterrents and Developer Resources
OpenAI is integrating new product-level interventions directly into ChatGPT. When a user attempts to find or generate abusive material, the system will now trigger a warning explaining the harm caused by sexualizing children, remove the flagged conversation, and pause the user's access to the service for 24 hours.

This image illustrates the specific product-level intervention and warning system OpenAI is integrating into ChatGPT to deter the generation of abusive material.
To support the broader ecosystem, OpenAI is offering the classifier to vetted organizations at no cost via its API. The company also published a CSAM guidance document to help developers build safety systems, covering topics such as reporting processes, detection tools, and support for human reviewers.
Law Enforcement and Industry Collaboration
The company is expanding its support for child-safety law enforcement by providing dedicated plugins and investigative tools. These tools are designed to help investigators analyze lengthy chat logs and legal process returns to identify connections and create timelines linked to source material.
OpenAI is also participating in cross-platform initiatives like the Tech Coalition's Lantern program to share and receive signals regarding online exploitation. These efforts are part of a broader commitment to coordinate with organizations like Thorn and the Internet Watch Foundation to strengthen industry-wide defenses.
What Hacker News readers are saying
- Experience: One commenter who reported a recent account ban claimed that OpenAI's actions are performative, alleging that their automated system was flagged despite being used for legitimate classification tasks. comment ↗
- Experience: A commenter with experience in the field suggested that companies may prioritize automated classifiers over nuanced approaches to avoid regulatory risks and to protect human workers from viewing disturbing content. comment ↗
- Experience: One user highlighted the practical benefits for law enforcement, noting that diffusion models have significantly increased the volume of content, making it harder to conduct forensic investigations on real cases. comment ↗
- Skeptical: Several commenters expressed skepticism regarding the motives behind these measures, suggesting they may serve as a pretext for increased surveillance and data collection. comment ↗
Summarized opinions from the Hacker News thread, not verified facts. Read the full discussion ↗