Telegram has often been cited as a haven for illegal activities, including the circulation of porn. In the latest case of porn-related problems on the messaging app, a group of researchers unearthed a rampant deepfake ecosystem on Telegram. Many deepfake bots available on Telegram generate fake nudes of women who use their photos on social media at the user’s request. These photos are then circulated on Telegram channels among users, sometimes for money.
The discovery of this ecosystem was led by researchers from a security company called Sensity. The company says it is dedicated to identifying visual threats such as the spread of deepfake technology. According to Sensity researchers, more than 100,000 images were exchanged on Telegram’s public channels in July this year. This statistic does not include images that were generated by deepfake bots but were never shared.
The scale of deepfake robots is unimaginably high due to the variety available for creating fake nudes on Telegram. The security company said that users of these Telegram channels, about 70%, belong to Russia and neighboring countries. Sensity said that although many channels have been disabled, a huge fraction of the deepfake ecosystem is still active. The Verge has independently verified and confirmed that these channels still exist. Sensity says that “a limited number” of fake nudes feature people “who appeared to be minors”.
While it is not clear who created these deepfake robots, they are available for free use by users. But there is a freemium model that is followed in the deepfake ecosystem. Deepfake bots will create fake nudes with watermarks and partial nudity when a user is not paying for the services. Users are required to pay a nominal fee to “uncover” these nudes with full exposure. According to the security company, the “beginner’s rate” is 100 rubles (about Rs 95) for 100 fake nudes without watermarks. This fee is valid for seven days.
The deepfake ecosystem uses software called DeepNude to generate fake nudes. DeepNude surfaced last June but was withdrawn hours later due to potential misuses of the software that would soon become common. Many media reports have highlighted this problem, forcing the creator to pull the plug from DeepNude. But the software never died, on the contrary, it reached backchannel. Sensity mentioned that DeepNude has been decrypted and much improved for open source repositories and torrents. The software is now being used on Telegram but in a more advanced form, where deepfake robots also make money by trading fake nudes.
Sensity said it reached out to Telegram to ask why the messaging app allowed the generation and circulation of the questionable content. Telegram has yet to respond. The security research firm said it also contacted law enforcement agencies.
The deepfake ecosystem on Telegram uses DeepNude software which was launched last year but was soon removed.
- A deepfake ecosystem on Telegram is creating hundreds of fake nudes.
- These fake nudes feature women without their knowledge.
- DeepNude is the software used on Telegram.