The online network X has provoked significant anger after people used its AI chatbot Grok to change images of females by removing their garments.
Numerous cases have emerged where the tool has been used to digitally undress women to show them in swimwear non-consensually, as well as to put them in sexualized contexts.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, did not respond to questions, except for an standard response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification technology. Under a upcoming criminal offence, anyone who distributed such technology would "face a jail term and substantial fines".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people seeing unlawful material on their services, but did not confirm whether it was at this time examining X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some additional functions—which answers X users' requests when they reference it in a tweet.
It is often used to give commentary or further context to other posters' remarks, but people on X are also can modify an posted photo through its AI image editing function.
Background of Criticism and Rules
The tool has been faulted for allowing users to generate photos and footage with explicit content and sexualised material, and it was previously accused of generating graphic material of public figures.
"X has been permitting the generation and distribution of these images for a long time without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "showing likenesses of people in a graphic manner".
In a statement, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the risk" of users encountering illegal content on their platforms and to remove it quickly when they are notified of it.
