Governments worldwide sounded alarms after xAI’s Grok chatbot generated tens of thousands of sexualized images of girls and women without their consent.
What happened: A wave of users of the X social network prodded Grok to produce images of public figures and private individuals wearing bikinis or lingerie, posed in provocative positions, and/or with altered physical features. Several countries responded by requesting internal data, introducing new regulations, and threatening to suspend X and Grok unless it removed its capability to generate such images. Initially X responded by limiting its image-alteration features to paying users. Ultimately it blocked all altered images that depict “real people in revealing clothing” worldwide and generated images of such subject matter in jurisdictions where it’s illegal.
How it works: Over a 24-hour period in late December, xAI’s Aurora, the image generator paired with Grok, produced as many as 6,700 sexualized images per hour, according to one analysis, Bloomberg reported. Grok typically refuses to generate nude images, but it complies with requests to show subjects of photographs in revealing clothes, The Washington Post reported. Several national governments took notice.
- Brazil: Legislator Erika Hilton called for Brazil’s public prosecutor and data-protection authority to investigate X and to suspend Grok and other AI features on X nationwide.
- European Union: German media minister Wofram Weimar accused Grok of violating the EU’s Digital Services Act, which prohibits non-consensual, sexual images and images of child sexual abuse as defined by member states.
- France: Government ministers decried “manifestly illegal content” generated by Grok while officials widened the scope of an earlier investigation into X to include deepfakes.
- India: The Ministry of Electronics and Information Technology demanded that X remove “unlawful content” and punish “offending users.” In addition, it ordered the company to review Grok’s technology and governance, address any shortcomings, and deliver a report to the government.
- Indonesia: The government blocked access to Grok in the country.
- Malaysia: Malaysia, too, blocked access to Grok following an investigation into X’s production of “indecent, grossly offensive, or otherwise harmful” images.
- Poland: Speaker of the parliament Wlodzimierz Czarzasty cited X to argue for stronger legal protections for minors on social networks.
- United Kingdom: The UK Home Office, which is responsible for law enforcement, said it would outlaw tools for “nudification.” The regulator for online platforms launched an inquiry into whether X had violated exsting laws.
- United States: Senators Ron Wyden (Oregon), Ed Markey (Massachusetts), and Ben Ray Luján (New Mexico), all of the Democratic party, sent open letters to the CEOs of Apple and Google requesting that they remove X’s app from their app stores, charging that X’s production of non-consensual, sexual images violates their terms of service.
X Responds: A post on X’s social feed for safety issues said the company will remove all posts containing images that depict (i) nudity without the subject’s consent and (ii) sexual abuse of children. Grok’s X account will no longer allow users paid or unpaid, in any jurisdiction, to alter images of real people to depict them in revealing clothing. In addition, Grok will prevent users from generating images of real people in bikinis or other revealing clothing where such images are illegal.
Behind the news: Governments have been trying to limit the use of image generators to satisfy male desires to see pictures of undressed women since around 2019, when an app for this purpose first appeared.
- In 2019 and 2020, the U.S. states of California and Virginia banned deepfakes that depict an individual’s “intimate body parts” or sexual activity with consent.
- In 2023, China enacted a law that requires strict labeling and consent for altered biometric data, including expression, voices, and faces, and the UK made sharing intimate deepfakes a priority offense.
- In 2025, South Korea criminalized possession and viewing of deepfake pornography, while the European Union’s AI Act required transparency for synthetic content.
- In the U.S., 2025’s Take It Down Act made it a crime to publish non-consensual “intimate” — typically interpreted to mean nude — imagery generated by AI.
Why it matters: Although other image generators can be used in the same way, the close relationship between X and Grok (both of which are owned by Elon Musk) adds a new dimension to regulating deepfakes. Previously, regulators absolved social networks of responsibility for unlawful material posted by their users. The fact that Grok, which assisted in generating the images, published its output directly on X puts the social network itself in the spotlight. Although the legal status of non-consensual “undressed” images (as opposed to nudes) is not yet settled, the European Commission could impose a fine amounting to 6 percent of X’s annual revenue — a warning sign to AI companies whose image generators can produce similar types of output.
We’re thinking: Digitally stripping someone of their clothing without their consent is disgusting. No one should be subjected to the humiliation and abuse of being depicted this way. In addition to Grok, competing image generators from Google, OpenAI, and others can be used for this purpose, as can Photoshop, although it requires greater effort on the user’s part. We support regulations that ban the use of AI or non-AI tools to create non-consensual, sexualized images of identifiable people.