OpenAI’s new AI tools aim to make online platforms safer for children Rachit Agarwal / Digital Trends
OpenAI is expanding its efforts to combat online child sexual exploitation with a new set of AI-powered detection tools, stricter interventions in ChatGPT, and additional support for developers and law enforcement agencies. The announcement, made on November 16, includes an image classifier designed to identify potential child sexual abuse material (CSAM), including AI-generated images, alongside measures intended to interrupt harmful behaviour before it escalates.
The move comes as generative AI makes it easier to create synthetic content, raising fresh challenges for platforms and organisations responsible for protecting children online. OpenAI says its latest efforts build on its existing Child Safety Blueprint and are intended to help organisations detect abuse, respond to reports and support investigations. The company is also working with child safety organisations and law enforcement rather than treating the problem as something individual technology platforms can solve alone.
OpenAI makes its CSAM detection tool available to vetted organisations
The new image classifier looks for visual indicators of potential abuse, including material that may be AI-generated. It assigns scores to images to help safety teams prioritise what needs human review.
