Kakao Brain announced the development of its newest face-swapping technology, ‘Smooth-Swap,’ designed to expand the capabilities of smooth face swapping through enhanced identity embedding[1]. The innovative model boasts its groundbreaking, simplified architecture after replacing complex external modules of existing systems with an integrated smooth identity embedder, which in turn facilitates faster, more stable face swapping.
Representing another important milestone in Kakao Brain’s face-swapping research, the company’s ‘Smooth-Swap: A Simple Enhancement for Face-Swapping with Smoothness’ paper will be presented at the upcoming global computer vision conference, CVPR 2022[2], for the second year in a row. This will include an exclusive oral presentation session reserved for the most outstanding papers among accepted articles (25.33% of 8,161 submissions were accepted this year). At last year’s event, only 4% of accepted papers were given time to do an oral presentation in which Kakao Brain was nominated for its exceptional research paper, ‘HOTR: End-to-End Human-Object Interaction Detection with Transformers.’ This year, not only has ‘Smooth-Swap’ managed to significantly reduce the complexity of its architecture, it also possesses great potential for commercialization, both of which have been recognized and rewarded by the premier computer vision conference.
An accurate and consistent identity gradient[3] is essential to seamlessly changing a person’s identity without sacrificing the image’s high quality. Trained via supervised contrastive loss, ‘Smooth-Swap’ acquires its stable identity gradient by learning embedding with a higher smoothness. These improvements address the earlier model’s weakness of adding handcrafted components and 3D face modeling which ultimately complicated its design and entailed sophisticated hyperparameter tuning. Instead, ‘Smooth-Swap’ relies on a simple U-Net-based architecture with an integrated smooth identity embedder to deliver cutting-edge performance.
Also Read: Sony AI’s Peter Wurman Inducted into the National Inventors Hall of Fame
The simple architecture and enhanced performance of ‘Smooth-Swap’ have not only made the technology competitive in terms of its commercialization potential and wider application, they also allow it to face more challenging face-swapping scenarios such as face swapping during video playback. ‘Smooth-Swap’ suggests a differentiated identity embedding approach and empowers the generator to create higher-quality images, especially when changing a subject’s face shape. Through Kakao Brain’s ‘Smooth-Swap,’ which enables fast and stable face swapping, it is expected to develop various kinds of digital humans such as virtual influencers, show hosts, and announcers.
“We are proud and excited to unveil the groundbreaking face-swapping technology, ‘Smooth-Swap,’ to the world,” said Kim Il-doo, CEO of Kakao Brain. “I strongly believe this technology will accelerate innovation in the face-swapping sphere, bringing us another step closer to the incredibly immersive metaverse we always dreamed of as wells as the digital human services of the future.”