California Launches Investigation into xAI Over AI-Generated Child Sexual Abuse Material and Nonconsensual Deepfakes
Key Points
- California Attorney General Rob Bonta announced an investigation into xAI over AI‑generated sexualized images of children and women.
- A report found more than half of 20,000 images generated by Grok between Christmas and New Year’s depicted people in minimal clothing, including minors.
- Governor Gavin Newsom called Grok a breeding ground for predators and urged action against nonconsensual deepfakes.
- The probe follows similar inquiries by the UK regulator Ofcom, EU officials, and blocks by Malaysia and Indonesia.
- xAI has placed rate limits on Grok’s image generation but has not fully shut down the feature.
- Elon Musk stated he was unaware of underage images and emphasized Grok’s compliance with the law.
- The case raises broader concerns about AI governance, content moderation, and protection of vulnerable groups.
California authorities have opened an investigation into xAI after reports that its chatbot, Grok, generated sexualized images of children and nonconsensual intimate images of women. Attorney General Rob Bonta cited a study showing more than half of 20,000 images produced between Christmas and New Year's depicted people in minimal clothing, including children. Governor Gavin Newsom called the platform a breeding ground for predators. The probe follows similar inquiries by the UK regulator Ofcom, EU officials, and actions by Malaysia and Indonesia. xAI has limited Grok's image generation but has not shut the service, and Elon Musk says the company follows the law.
Background of the Investigation
California Attorney General Rob Bonta’s office announced a formal investigation into xAI, the company behind the chatbot Grok, after a series of reports indicated the system was creating sexualized images of children and nonconsensory intimate images of women. The investigation was triggered by a report that more than half of the 20,000 images generated by Grok between Christmas and New Year’s depicted people in minimal clothing, some appearing to be children.
Official Statements and Government Response
In a statement, Bonta emphasized the state’s zero‑tolerance policy for AI‑generated child sexual abuse material (CSAM) and nonconsensual intimate images, describing the alleged activity as “vile.” Governor Gavin Newsom supported the Attorney General’s move, calling Grok a “breeding ground for predators” that spreads nonconsensual sexual deepfakes, including digitally undressed minors.
International Regulatory Context
The California probe is not the first. The United Kingdom’s regulator Ofcom has opened its own inquiry, and European Union officials have indicated they are examining the issue as well. Additionally, Malaysia and Indonesia have taken steps to block Grok, reflecting broader global concern about AI‑generated illicit content.
xAI’s Response and Mitigation Efforts
In reaction to the mounting pressure, xAI recently imposed rate limits on Grok’s image‑generation capabilities but has not completely disabled the feature. When asked to comment on the California investigation, the company replied with an automated email stating “Legacy Media Lies.”
Elon Musk’s Position
Elon Musk, who leads xAI, said he was “not aware of any naked underage images generated by Grok.” He emphasized that Grok’s operating principle is to obey the law and that the company works to address cases of “adversarial hacking of Grok prompts.”
Implications for AI Governance
The investigation highlights growing scrutiny of generative AI tools that can produce harmful content. It underscores the tension between rapid AI development and the need for robust safeguards to prevent misuse, especially when vulnerable populations such as children are at risk.