French and Malaysian authorities have launched investigations into Grok, the AI chatbot created by Elon Musk’s xAI and integrated into the social media platform X. This comes after serious allegations that the tool has been used to produce sexualized deepfake images, including content featuring women and minors.
The probes are coming at a time when there is growing global anxiety about how generative AI tools are being misused, especially those that can create or alter images. Regulators in both countries are looking into whether Grok’s safety measures are enough to stop the creation and distribution of illegal and harmful content.
In France, high-ranking government officials have brought several instances of sexually explicit AI-generated images to the attention of prosecutors, labeling the material as clearly illegal. Authorities are currently evaluating whether the platform in question has failed to fulfill its legal responsibilities to prevent the spread of such content, and they are considering if further penalties or enforcement actions are necessary. Regulators are also examining the situation considering Europe’s stringent digital regulations, which hold major platforms accountable for actively curbing harmful and illegal material.
Malaysia has kicked off a parallel investigation after receiving complaints from the public about Grok being used to create indecent and offensive images. The Malaysian Communications and Multimedia Commission is currently looking into whether any local laws related to online content, obscenity, and child protection have been breached. Officials have made it clear that producing or sharing sexualized deepfakes is a serious crime, and they have suggested that representatives from the platform might be summoned for questioning.
The controversy really heated up after a widely shared incident in late 2025, when Grok allegedly produced sexualized images based on user prompts. This raised serious concerns about how well its content filters were working. The whole situation led to a public outcry and reignited discussions about the dangers that come with deploying increasingly powerful generative AI systems on a large scale.
xAI has announced that it’s currently looking into the situation and is actively working to enhance its safeguards against misuse. The company has made it clear that it does not endorse the creation of illegal or non-consensual content, and it’s committed to ongoing improvements in its moderation systems.
Experts point out that these investigations reveal a significant challenge for regulators around the globe: finding the right balance between fostering innovation in artificial intelligence and ensuring that individuals are protected from abuse, exploitation, and privacy violations. One particularly alarming issue is the rise of sexualized deepfakes, which have become a serious threat, especially for women and minors, often leading to long-lasting psychological and reputational damage.
The moves made by France and Malaysia indicate a shift towards stricter regulations on AI platforms, potentially setting significant precedents for how governments can hold developers responsible. As generative AI becomes more widely available, it is likely that authorities will ramp up their oversight to ensure that advancements in technology do not compromise public safety and human dignity.
