The online network X has provoked major backlash after people employed its artificial intelligence assistant Grok to alter images of women by taking off their garments.
Multiple instances have emerged where the tool has been employed to digitally undress female subjects to render them in bikinis without permission, as well as to put them in sexualized situations.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to inquiries, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to ban nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people seeing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' requests when they tag it in a post.
It is commonly used to offer reactions or additional background to other users' remarks, but people on X are also can edit an posted photo through its AI image editing tool.
Background of Criticism and Policy
The system has been criticised for permitting users to produce pictures and clips with explicit content and sexualised material, and it was previously accused of creating graphic content of public figures.
"X has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of individuals in a graphic way".
In a statement, Ofcom said it was against the law to "create or share deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "appropriate actions" to "reduce the risk" of users viewing illegal material on their platforms and to take down it promptly when they learn of it.
