AI sex content leaves real trauma among victims, says activist

LifeUp Malaysia's Liew Li Xuan says debates often focus on the spread of sexual content rather than the emotional and psychological impact on victims.

AI sexual deepfakes are more than a tech problem, say activists, who highlight the legal gaps and the risks of normalising non-consensual content.

PETALING JAYA:

 Conversations around AI-generated sexual content in Malaysia often overlook the real harm inflicted on victims, particularly young girls, an activist said, amid global backlash over such material created by chatbots.

Liew Li Xuan, co-founder of digital wellbeing NGO LifeUp Malaysia, said discussions typically centre on how widely the content spreads rather than its emotional and psychological impact.

She said that victims experience real trauma even though the images generated are fake, and have to relive it when reporting the matter to authorities or parents.

“They suffer from anxiety, shame and social withdrawal as well as disruption to school life,” she told FMT. Many victims remained silent out of fear and embarrassment, allowing abusive content to circulate unchecked in private group chats and online spaces.

Her comments come after international criticism of Elon Musk’s Grok artificial intelligence chatbot on social media platform X, which has been repeatedly misused to generate sexually obscene, offensive and non-consensually manipulated images.

Several governments and regulators have raised concerns over weak safeguards after reports that the tool generated manipulated images of women and minors, prompting calls for stronger oversight and accountability.

In Malaysia, the Malaysian Communications and Multimedia Commission has ordered a temporary restriction on access to Grok.

Beyond a tech problem

Suyasha Srii Sukendrran, co-director of Pocket of Pink Education, said AI sexual abuse should be treated as a matter of consent and gender-based violence, not just a technological problem.

“Without social, legal, or familial feedback, the perpetrator might think this is acceptable and that no one gets hurt,” she said, adding that meme culture has normalised the sharing of images without consent.

Suyasha warned that the lack of consequences could encourage perpetrators to see non-consensual AI sexual content as acceptable and repeat the behaviour.

“This can normalise objectifying or controlling others, which increases the likelihood of future sexual misconduct if left unaddressed,” she said.

Legal gaps remain

Lawyer Azira Aziz said that platform liability for AI sexual harm remains largely untested in Malaysia. “There is yet to be a class action suit against social media platforms regarding the misuse of AI strictly in the context of gender-based violence,” she said.

“It remains to be seen whether the Attorney-General’s Chambers is willing to take it up against giants like X.” She added that AI sexual deepfakes of adult women could fall under existing offences such as obscenity and insult to modesty. - FMT