GoPlus security research: Beware of the dangers of AI face-changing

Original title: "GoPlus Security Research: Beware of the Hazards of AI Face Swapping"

Original source: GoPlus

1. What is "AI Face Swapping"

AI face swapping usually refers to the use of deep learning Deepfakes technology to replace one person's face with another person's face. This technology can create fake videos or pictures that look real by analyzing the facial features of the target person and then applying them to another video or picture.

Face swapping technology has certain application value in the fields of entertainment and filmmaking, but it also brings many hazards. Specifically, Deepfakes has received widespread attention for its potential use in the production of child sexual abuse materials, celebrity pornographic videos, revenge porn, fake news, pranks, bullying and financial fraud.

2. The history of “AI face-changing”

In 2017, Supasorn Suwajanakorn and others from the University of Washington used deep learning technology to launch the “Synthesizing Obama”[1] project, which was published in Siggraph 2017. In short, this technology modifies Barack Obama’s video clips and makes him read words from a new audio track that is unrelated to the video. The technology is a comprehensive application of technologies such as Recurrent Neural Network (RNN) and Mouth Synthesis. The effect is very good. The audio and mouth in the video are matched to achieve a realistic synthesis effect:

Technical overview of synthesizing obama

Another similar project is Face2Face proposed by Justus Thies, a professor at TU Darmstadt in Germany in 2016 [2] (this technology modifies the facial video clips of the target person (dst: destination) so that it can imitate the facial expressions of the person in the video (src: source) in real time, published in CVPR2016). Because of this project, Justus Thies was also received by German Chancellor Merkel in 2019 to discuss the risks and challenges of media tampering [3].

The arrow points to Justus Thies

The above technologies prove that the foundation of deep fake technology based on deep learning is already in place. So, where did deepfakes come from?

The answer is: Reddit!

Yes, the term Deepfakes originated around the end of 2017, by a Reddit user named “deepfakes”. [4] He created the r/deepfakes channel in the Reddit community, where everyone shared the Deepfakes videos they created: the most popular one at the time was to replace the faces of celebrities with the bodies of actors in pornographic videos.

At almost the same time, various videos and communities emerged like mushrooms after rain. Everyone shared their videos in the forum. Among them, many visual effects post-production workers added various complex special effects and super-resolution technologies to the face-changing videos, making their quality reach the real Photo-realistic effect, and widely disseminated on UGC video platforms such as Youtube. According to research statistics, the total number of visits to Deepfakes-related videos on platforms such as Youtube, Instagram, and Tik-tok has exceeded 10 billion+. A single video of a related high-quality video producer has exceeded 1 million hits.

Behind these videos, we cannot ignore the most successful face-changing software in the open source community, the DeepFaceLab series [5,6] (total Github 66,000+ stars, top 0.0001% of Github, one of the top ten AI open source projects in 2020, along with Tensorflow PyTorch), which was developed by Ivan Perov and Daiheng Gao (Daiheng is also the AI Advisor of GoPlus, focusing on the prevention and treatment of PoH) and others.

DeepFaceLab

Compared with academic technologies, a series of open source deepfakes technologies represented by DeepFaceLab and FaceSwap have the outstanding advantage of being easy to use and can be used without technical background!

Different from traditional Github projects, DeepFaceLab provides a one-click script bat based on Windows Linux platform, that is, users can follow the process to train and reason about face-changing models, which greatly reduces the threshold for making face-changing technology. This is also the key to why face-changing videos can be promoted and spread!

3. The harm of "AI face-changing"!

· Cases in the traditional financial field:

In the traditional financial field, face-changing technology is used for fraudulent activities. For example, according to reports [7,8], criminals use illegally obtained personal information to simulate the faces and voices of the victims’ relatives, leaders, colleagues, or public officials through computer algorithms, and impersonate the above-mentioned people to commit fraud. After gaining the victim’s trust, they use pre-prepared routines to send the victim fraud information such as bank card transfers, virtual investment and financial management, and rebates for order swiping, and use video calls and voice bombing to further reduce the victim’s vigilance, making it difficult for the victim to detect abnormalities in a short period of time. Once the victim believes the fraudster’s tricks and completes the transfer, the other party disappears without a trace.

· Web3 Case

Recently, everyone has paid attention to “AI face-changing” again, because the user’s OKX digital currency was added to the address whitelist by someone using AI face-changing, thereby transferring funds. Worth more than 3 million US dollars! [9]

The news broke on Monday (June 3, 2024), causing concerns among many web3 users about the security of facial recognition verification!

Previously, Web3 users were not sensitive to facial verification in the Proof-of-Human process because it was rarely used. After the huge financial losses caused by the face-swapping issue broke out, many people began to doubt whether the KYC and facial verification technology of large exchanges such as OKX and Binance are reliable.

Is there a fair and reasonable third-party organization to verify it and judge the security of these large exchanges?

4. New technological advances in 2024 and warnings to users

In 2024, two weeks after the emergence of OpenAI, EMO [10] (Daiheng Gao's group) from Alibaba DAMO Academy was born, and the long-term face-driven technology based on the diffusion model was declared mature.

This means that criminals can make high-quality fake face videos with just a single photo to deceive the exchange's verification channel!

In addition, with the maturity of voice cloning technology (bert vits2 [11]), criminals with ulterior motives deliberately collect user audio information to use the model to replicate the human voice required for exchange verification!

It can be seen that the new developments in technology in 2024 require users and exchanges to be more vigilant. Unfortunately, the current Web3 industry does not understand this type of technology, and lacks relevant talents to contribute to targeted changes to effectively protect the interests of users.

As users, we can only ensure that our personal information is leaked as little as possible, and resolutely not record face-revealing videos with sound and pass them around on the Internet to avoid potential risks.

5. GoPlus in action

Since establishing a partnership with Daiheng Gao in March 2024, GoPlus has continued to support Daiheng's research in the field of video face forgery and has funded related laboratories (USTC Cybersecurity Research Institute, etc.).

In the field of images, Daiheng won the second place in the Deepfake detection challenge held by Meta in 2019. Currently in the field of video, Deepfake detection is a more noteworthy part!

User property safety is the lifeline. GoPlus is willing to work with users to provide state-of-the-art solutions for exchanges, KYC, etc., and use the most advanced AI technology to do our utmost to reduce the occurrence of risks!

References: [1] Suwajanakorn, Supasorn, Steven M. Seitz, and Ira Kemelmacher-Shlizerman. "Synthesizing obama: learning lip sync from audio." ACM Transactions on Graphics (ToG) 36.4 (2017): 1-13. [2] Thies, Justus, et al. "Face2face: Real-time face capture and reenactment of rgb videos." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. [3] Cabinet Meeting: Synthetic Media - Danger or Opportunity? [4] Cole, Samantha (24 January 2018). "We Are Truly Fucked: Everyone Is Making AI-Generated Fake Porn Now". Vice. Archived from the original on 7 September 2019. Retrieved 4 May 2019.

[5] DeepFaceLab, https://github.com/iperov/DeepFaceLab

[6] DeepFaceLive, https://github.com/iperov/DeepFaceLive

[7] How to prevent “AI face-changing” fraud: Tips from financial regulators https://www.gov.cn/lianbo/bumen/202310/content_6907773.htm

[8] In-depth: You can know a person’s face but not his heart! Beware of “AI face-changing” financial fraud https://m.21jingji.com/article/20231020/herald/1f23a5a3959ab7cc544e0c0d93ed6886.html

[9] AI face-changing bypasses OKX censorship system, users lose $3 million https://x.com/BroLeonAus/status/1797553316404371967

[10] EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions https://humanaigc.github.io/emote-portrait-alive/

[11] Bert Vits2 https://github.com/fishaudio/Bert-VITS2

This article is from a contribution and does not represent the views of BlockBeats.

欢迎加入律动 BlockBeats 官方社群：

Telegram 订阅群： https://t.me/theblockbeats

Telegram 交流群： https://t.me/BlockBeats_App

Twitter 官方账号： https://twitter.com/BlockBeatsAsia