The growing accessibility of generative artificial intelligence has triggered serious concerns in India, as women increasingly report their photographs being misused to create sexually explicit images without consent — a form of digital abuse that experts warn is difficult to reverse once it spreads online.
A single prompt such as “remove dress” or “bikini image” can now transform an ordinary photograph into explicit content, often driving women away from sharing images on social media altogether. The issue has gained renewed attention following multiple incidents involving X and its AI tool Grok, where users openly tagged the chatbot under women’s photos, urging it to make sexually suggestive alterations.
While Grok itself is governed by moderation rules, the public nature of such requests has normalised harassment, critics say. Women have described feeling violated as their images are turned into fodder for lewd fantasies masked as “tech curiosity”, with platforms often responding slowly.
Political and official response
Shiv Sena (UBT) MP Priyanka Chaturvedi has written to the Ministry of Electronics and Information Technology, seeking safeguards against such misuse. Following multiple incidents, including cases involving minors, the Centre issued notices to X in December, asking it to explain lapses in content moderation.
In a related case in Hyderabad, police arrested a security guard for secretly photographing women and using AI tools to generate manipulated images, underscoring that the threat extends beyond high-profile platforms.
Beyond social media platforms
Messaging app Telegram has emerged as a hub for AI-based image morphing, with bots openly advertising “clothing removal” services. Experts say the anonymity and limited moderation on such platforms raise serious concerns, especially about the exploitation of minors.
Even more troubling is the rise of offline AI models that can be run on personal computers, bypassing platform oversight entirely and complicating law enforcement efforts.
Legal gaps and expert concerns
Cyber law expert Rupesh Mittal said such acts qualify as deepfakes and image-based sexual abuse under existing laws. He pointed to provisions under the IT Act and the Indecent Representation of Women (Prohibition) Act but noted enforcement challenges, especially when offenders operate anonymously.
The Internet Freedom Foundation has cautioned that proposed regulations on synthetic content must avoid overreach that could harm free expression, while still addressing clear harms.
As AI tools continue to evolve, experts stress the need for faster grievance redressal, stronger compliance requirements for platforms, and clearer legal accountability to ensure technology does not become a weapon against personal dignity.