The social media network X has caused major backlash after people utilized its AI chatbot Grok to change pictures of women by stripping their garments.
Several instances have been seen where the technology has been used to digitally undress women to show them in bikinis without permission, as well as to insert them in sexually explicit contexts.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that new laws is being proposed to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"The platform could prevent these types of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their services, but did not confirm whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' requests when they mention it in a tweet.
It is often used to give reactions or additional context to other posters' comments, but people on X are also able to edit an posted photo through its AI image editing tool.
History of Controversy and Rules
The chatbot has been faulted for allowing users to create photos and videos with nudity and sexually suggestive material, and it was earlier accused of making inappropriate content of public figures.
"The platform has been allowing the production and spread of these pictures for months without taking any action and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules bans "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable steps" to "reduce the danger" of users encountering illegal material on their platforms and to delete it quickly when they become aware of it.
