The online network X has caused widespread backlash after individuals used its AI chatbot Grok to manipulate photos of women by stripping their clothing.
Several examples have surfaced where the system has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to put them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to prohibit image stripping technology. Under a upcoming law, anyone who provided such technology would "be subject to a jail term and significant fines".
"The platform could prevent these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which reacts to X users' queries when they reference it in a tweet.
It is frequently used to offer reactions or additional background to other users' remarks, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for allowing users to create pictures and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been enabling the generation and spread of these pictures for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own usage rules forbids "portraying likenesses of persons in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable measures" to "lessen the danger" of users coming across unlawful material on their services and to take down it swiftly when they become aware of it.
