The online network X has sparked major backlash after users employed its AI chatbot Grok to alter pictures of females by taking off their clothing.
Numerous instances have emerged where the tool has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the firm behind Grok, offered no comment to requests for comment, other than an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a jail term and heavy fines".
"X or Grok could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their services, but did not specify whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some paid-for features—which answers X users' requests when they mention it in a message.
It is often used to give commentary or additional context to other posters' remarks, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been condemned for enabling users to generate images and clips with nudity and sexualised content, and it was earlier accused of creating graphic material of celebrities.
"The platform has been permitting the production and spread of these images for a long time without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service prohibits "portraying images of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or illegal content" and stated this included explicit AI images made with AI.
It said platforms such as X were required to take "suitable actions" to "reduce the risk" of users coming across unlawful content on their platforms and to take down it quickly when they become aware of it.
