Newshot

OpenAI launches child sexual abuse content detection tool to strengthen child online safety protections

AI summary

OpenAI announced the launch of a child sexual abuse material (CSAM) detection tool that can interrupt harmful behavior, provide interventions, and support law enforcement investigators handling child safety cases. The tool will be offered free of charge through the OpenAI API to vetted organizations. OpenAI has introduced a new image classifier that looks for visual features of abuse to help moderation teams identify previously unseen images, and it reduces falsely flagged images by 48% compared with existing safeguards. In addition, OpenAI will add product warnings in ChatGPT: when users attempt to seek sexual abuse material, the system will interrupt the action and direct users to help, and the account will be temporarily suspended for 24 hours, with repeat violations leading to longer suspensions and eventual permanent bans or reports to authorities.

Translation(model translation, not the source's own text)

OpenAI Launches Child Sexual Abuse Content Detection Tool to Strengthen Child Online Safety Protection

IT Home, November 17 news: OpenAI yesterday announced the launch of a child sexual abuse material (CSAM) detection tool. The product can interrupt harmful behavior while providing intervention measures and supporting law enforcement investigators handling child safety cases. The tool will be made available free of charge through the OpenAI API to vetted organizations.

According to the introduction, OpenAI has launched a new image classifier for detecting potential child sexual abuse material. The classifier looks for visual characteristics of abuse to help review teams identify never-before-seen images, while also providing scoring functionality to help reviewers determine which images may need to be prioritized.

The National Center for Missing & Exploited Children (NCMEC) provided support during the tool's development, assisting OpenAI with training, evaluation, and actual deployment.

At the same time, compared with existing protection measures, the tool can reduce the number of falsely flagged images by 48%. Among flagged images, the likelihood of actually containing sexual abuse content is approximately twice as high as before. invideo AI testing showed that one in every four flagged cases was ultimately banned.

In addition, OpenAI will add product warnings in ChatGPT. When users attempt to seek sexual abuse material, the system will interrupt the relevant operation, explain the consequences, and guide users to seek help in order to stop harmful behavior. Once a user makes such a request, their ChatGPT account will be temporarily suspended for 24 hours; repeated violations may lead to longer suspensions, and ultimately permanent bans or reports to relevant authorities.

OpenAI has partnered with Anthropic and Google to jointly commission the development of a new guide on how to respond to disclosures of child sexual exploitation and abuse in AI systems. The guide was recently published by Tech Coalition, providing practical recommendations for developers handling these sensitive situations.

Related Articles

  • Overclocker uses AI to modify motherboard BIOS: attempts to revive ASRock SkyOC BCLK overclocking feature, bypassing Intel's non-K processor restrictions
  • OpenAI 奥尔特曼 talks about 黄仁勋's departure: I've been fired too, and I believe the AI industry can get through it smoothly
  • California formally sues OpenAI, alleging AI agents went out of control at least 80 times
  • OpenAI expands ChatGPT academic research free program: 50,000 more spots open on November 23, changing from lottery to first-come-first-served
  • OpenAI reaches AI safety partnership with Goodfire, after previously discussing an acquisition worth over $5 billion
  • Reserve Bank of Australia study: AI hits young workers first, especially in software and customer service roles

The translation is machine-generated and individual terms may be off. Nothing is rewritten or cut; the original is authoritative.

Published by IT之家
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗