Researchers map AI facial tracking to static images with scaringly accurate results

Innovators have come up with an extremely effective way to realistically manipulate the facial features of people. All that is needed is to do this is a webcam and a still image.
MarioNETte gives a whole new meaning to the term face swap. Photo: HyperConnect

A spitting image

South Korea-based researchers at Hyperconnect have created a tool called MarioNETte. This application uses state-of-the-art machine cutting techniques to reenact a face to near-perfect proportions. Venture Beat reports that the company aims to synthesize a reenacted face animated by the movement of a person, which is the “driver” while preserving the appearance of the “target”.

Altogether, this is not a new concept as other companies have experimented with this process. However, current firms have come across struggles to create a seamless reenactment. This is because unlike MarioNETte, tracking tools would usually require a few minutes of training data and could only reenact predefined targets. Along with this, the target’s features would often become distorted.

MarioNETte 2
The swift process of MarioNETte produces results that can be quite surprising. Photo: HyperConnect

Improvements still to be made

By looking at MarioNETte’s images, the replication looks a cut above some of the current key reenactment tools. However, the firm has admitted that more work needs to be done for the technology to be more robust.

“[Our] proposed method [does] not need [an] additional fine-tuning phase for identity adaptation, which significantly increases the usefulness of the model when deployed in the wild,” states a preprint paper that detailed MarioNETte’s architecture and validation, as reported by Venture Beat.

“Our experiments including human evaluation suggest the excellence of the proposed method.”

Controversial industry

This technology would be able to reimagine historical figures, celebrities, politicians and public figures. This would be useful for moviemakers when creating documentaries or biopics. Many creatives would also be tempted to use this when bringing back a movie saga but the actor in the main role has passed away.

MarioNETte Process
How the mapping process connects the driver and the target. Photo: HyperConnect

There is a huge debate about the ethics behind this process. Moreover, many actors such as Robin Williams had already made legal movements to prevent things this from happening to them. However, one of the biggest concerns regarding this technology is its use for hostile purposes.

Deepfakes have been a topic of controversy over the last year. This manipulation trend is often experimented with for trivial purposes. Although, there are several people that have dangerous motives. Many people have been using this kind of tech to implicate others and put them in vulnerable situations.

MarioNETte Images
The transition of imagery with MarioNETte. Photo: HyperConnect via Arxiv

Subsequently, many social media firms have started to revise their policies regarding Deepfakes, in order to prevent sensitive situations from occurring.

Therefore, with reenactment programs becoming further refined, more people could be dangerously mimicked. Nonetheless, this is a breakthrough in AI technology, shaping up a strong decade ahead for the industry.

What do you think about the advancement of AI facial tracking to reenact humans? Let us know your thoughts in the comment section.

0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments
You May Also Like