The social media network X has sparked major outrage after users employed its AI chatbot Grok to alter pictures of women by removing their clothing.
Several instances have surfaced where the technology has been used to virtually strip female subjects to render them in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization behind Grok, did not respond to questions, other than an auto-reply message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to ban nudification tools. Under a new criminal offence, anyone who distributed such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their services, but did not confirm whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' prompts when they mention it in a post.
It is frequently used to offer reactions or further background to other users' comments, but people on X are also can modify an posted picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been criticised for permitting users to generate pictures and videos with nudity and sexualised material, and it was previously accused of creating graphic content of celebrities.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of persons in a graphic fashion".
In a statement, Ofcom said it was unlawful to "make or share private pictures or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "necessary actions" to "minimize the risk" of users encountering illegal material on their platforms and to take down it swiftly when they are notified of it.
