The online network X has caused major outrage after individuals used its AI chatbot Grok to alter images of females by removing their clothing.
Numerous cases have been seen where the system has been used to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Stance
XAI, the company that created Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who provided such tools would "face a jail term and significant fines".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing unlawful material on their services, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium functions—which reacts to X users' requests when they reference it in a tweet.
It is frequently used to offer commentary or further background to other posters' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been faulted for allowing users to generate photos and footage with nudity and sexualised material, and it was previously accused of creating sexually explicit material of celebrities.
"The platform has been enabling the generation and distribution of these images for months without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own usage rules prohibits "portraying images of persons in a graphic manner".
In a comment, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable measures" to "minimize the risk" of users viewing illegal content on their services and to remove it promptly when they become aware of it.
