Photo: Tada Images / Shutterstock.com
Google’s generative AI received 258 user reports about suspected AI-generated extremist material and 86 reports related to AI-generated child sexual exploitation and abuse (CSAM) material between April 2023 and February 2024. Google has not confirmed how many cases are verified, casting doubt on the effectiveness of its content moderation systems. Furthermore, big tech companies such as Google, Meta, WhatsApp, X, Telegram, and Reddit lack appropriate measures and response time while dealing with harmful content.
A report by Australia’s eSafety Commissioner exposes critical discrepancies in how platforms handle TVE (Terrorist and Violent Extremist) content. WhatsApp, for instance, does not prohibit all organisations on Meta’s Dangerous Organisations and Individuals list from using its private messaging service, potentially allowing extremist organisations to operate unchecked.
“This discrepancy may mean that terrorist and violent extremist organisations are able to operate on parts of WhatsApp without action taken against them by the service,” said eSafety Commissioner Julie Inman Grant.
Additionally, Google relies solely on hash-matching technology to detect exact TVE content matches on YouTube and Google Drive, despite the availability of tools that can identify altered versions. As per the report, this approach is inadequate, particularly considering Meta’s statement that over 800 different versions of the Christchurch attack video circulated online shortly after the incident.
Meta’s Messenger, Telegram, and WhatsApp do not have measures to detect live-streamed TVE, despite the Christchurch attack being broadcast on Facebook Live in 2019. Moreover, Facebook and YouTube users who are not logged in cannot report live-streamed extremist content, delaying crucial interventions.
“We are concerned about the safety deficiencies that still exist in Facebook today, with users unable to report livestreamed TVE in-service if they are watching without being logged in. A user not logged in to YouTube faces the same hurdle,” said the eSafety Commissioner.
Another concerning matter is that the response time for user reports on TVE content remains slow. WhatsApp took over a day to address reports, Reddit averaged 1.3 days, and Threads took 2.5 days. Telegram reported an 18-hour response time for reports in Charts and Secret Chats.
The report also reveals that while Telegram detected previously removed TVE content through hash-matching, it does not use external databases such as the Global Internet Forum to Counter Terrorism (GIFCT). Additionally, Telegram lacks proactive content in private groups and private channels, limiting its ability to curb extremist propaganda effectively.
In the News: Silk Typhoon exploits zero-day vulnerabilities to target IT supply chain
