Impersonate Anyone With This Technique | Two Minute Papers #254

Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Two years ago, in 2016, we talked about a
paper that enabled us to sit in front of a camera, and transfer our gestures onto a virtual
actor. This work went by the name Face2face and showcased
a bunch of mesmerizing results containing reenactments of famous political figureheads. It was quite amazing. But it is nothing compared to this one. And the reason for this is that the original
Face2face paper only transferred expressions, but this new work is capable of transferring
head and torso movements as well. Not only that, but mouth interiors also appear
more realistic and more gaze directions are also supported. You see in the comparisons here that the original
method disregarded many of these features and how much more convincing this new one
is. This extended technique opens up the door
to several, really cool new applications. For instance, consider this self-reenactment
application. This means that you can re-enact yourself. Now, what would that be useful for, you may
ask? Well, of course, you can appear to be the
most professional person during a virtual meeting, even when sitting at home in your
undergarment. Or, you can quickly switch teams based on
who is winning the game. Avatar digitization is also possible, this
basically means that we can create a stylized version of our likeness to be used in a video
game. Somewhat similar to the Memoji presented in
Apple’s latest keynote with the iPhone X. And the entire process takes place in real
time without using neural networks. This is as good as it gets. What a time to be alive! Of course, like every other technique, this
one also has its own set of limitations. For instance, illumination changes in the
environment are not always taken into account, and long-haired subjects with extreme motion
may cause artifacts to appear. In short, don’t use this for rock concerts. And with this, we are also one step closer
to full character re-enactment for movies, video games and telepresence applications. This is still a new piece of technology, and
may offer many more applications that we haven’t thought of yet. After all, when the internet was invented,
who thought that it could be used to order pizza or transfer Bitcoin? Or order pizza and pay with bitcoin. Anyway, if you have some more applications
in mind, let me know in the comments section. Thanks for watching and for your generous
support, and I’ll see you next time!

79 thoughts on “Impersonate Anyone With This Technique | Two Minute Papers #254

  1. Wow, this opens up a whole range of possible moral / legal questions. It'll be interesting to see how the law adapts to these types of technology in the next few years. I suspect it won't be able to – technology just moves too quickly for a law system that wasn't designed to be fast or efficient

  2. It could potentially be really useful for the clothing companies, you could literally stand in front of a digital mirror and switch clothes instantaneous and not only that the depth camera could also be used to measure your body size to guarantee that you'll always get the correct sizes delivered once you click the buy button.

  3. … won't this make impersonation trivially easy? Wire this to a TTS trained on the victim's voice, voice recognition to feed the TTS system, and an actor, then you can make the victim say anything and have it recorded on camera.

  4. How do we stop people trying to start wars with this stuff? There was an entire season of 24 based on a fake voice recording now we'll have fake videos of our most loved figures being assholes.

  5. Hi! ᵗʰᶦˢ ᶜᵒᵐᵐᵉⁿᵗ ʷᵃˢ ᵃᵘᵗᵒᵐᵃᵗᶦᶜᵃˡˡʸ ᵍᵉⁿᵉʳᵃᵗᵉᵈ ᵇʸ ᵃ ᵇᵒᵗ

  6. Have you seen the 1987 movie Running Man starring Arnold Schwarzenegger? IMDB plot description: "A wrongly convicted man must try to survive a public execution gauntlet staged as a game show."

    [Spoiler alert!]

    Against all odds he's winning the staged death match, so the broadcasters use computers to switch the appearance of the two contestants on TV. I remember thinking "Yeah, right! That's a cool idea, but it would never be possible." Little did I know what the future would bring.

  7. Catfishing just moved into an entirely new level. I do seriously worry about the implications this has towards the (further) weaponization of fake news tho… I don’t trust the right with this kind of power.

  8. Honestly, you are one of the best channels on youtube. I'm always hyped when I see your video's notifications and you never let me down. Thank you and keep up the good work

  9. Amazing. Actors are now for all practical purposes essentially immortal! AND PRON WHAT A TIME PAF!

  10. it doesn't look bad but can still very clearly be distinguished as fake. Who knows maybe put a ConNet to use for preparation of a more realistic image becasue c mon how could we ever accept something these days without a neural netowork 😀

  11. I think live cam porno could take a hit when Amy is revealed to be in fact a 65 year old overweight electrician called Barry, but i don't care I'm in love!

  12. It could create new generation of theaters
    , like, i would edit my face as hero's face and so i could watch myself killing stuff in slow motion

  13. With this we can bring to live dead actors and actresses, as Carrie Fisher for the next Star Wars movie… or bring them to a younger version of themselves… the possibilities for movies are awesome!

    With this in fact, an amateur could do an amateur movie with famous actors and do all the acting him or herself! Is like taking the motion capture and digital actors to a next level.

  14. Another application for this is helping people with social anxiety gain a more friendly and sociable vision of themselves, where a more sociable and confident person controls the avatar interacting and speaking and presenting to others in a more confident way than what the SA person is used to seeing in themselves.

  15. The popup thing at the bottom said you need a camera with depth information. Does this apply to the video of the person doing the re-enactment, the original target actor video, or both?

  16. Is there anything for faking voices? I think that would be a million times more interesting if we could fake voices and bodies, or change voices to anything, it would be super useful too.

  17. It’s so irresponsible to discuss this technology without mentioning the obvious and immediate dangers to society. In fact, the dangers are much more obvious and likely to come into play than the silly use cases mentioned. I’m surprised that an educational channel wouldn’t address this truthfully. Yuck.

  18. Ordering pizza and transferring digital money via internet would be easy to predict tbh.

  19. The whole process of video is far more better than Apple WWDC18, at least not 5 years old cartoon face

  20. starts playing video "This is Two Minute papers with Károly-" My five year old daughter comes running down the stairs. "Wait! These are my favorite videos in the whole world! Don't watch them without me!"

  21. I'll bet it will be used for porn. Configure your fantasy body type on the fly, and the puppeteers are who-knows-what. Real webcam acts will not be able to compete.

  22. I am curious as to how this will play out in the long run. At the moment I'm imagining all politicians growing out their hair and making quick head movements all the time so they can't be copied as easily.

  23. Oh wao with this actually actors can act from home, they need not be in the sets. Secondly they need not wear uncomfortable clothes or other garments all the time, specially in some mythological and time serials/movies there are some garments that are uncomfortable, they need not wear it all the time.

    And did I hear it right there is no neural network used here?

  24. So there are plenty of use cases for this tech to enrich people's lives in trivial ways, such as those described in the video, but there are far more scary consequences when it is used to create false or misleading news that is all the more believable thanks to the video "evidence". Since the general population, and to some extent, the legal system, tends to be much further behind the curve in terms of knowing about cutting edge research than those interested in tech, there is a large window of opportunity before people know this is even possible, and where all kinds of chaos can be caused by fake videos. Even being accused falsely of a serious crime can ruin people's lives, but if there is faked video evidence, it will become much harder for people to escape being permanently tarnished by a false story about them, even if they are later cleared of any charges, since retractions to false stories and not guilty verdicts are much less visible in the media than negative stories containing accusations and trials.

    It may be easy to tell the video from actual recordings now, but I don't doubt that this technology could advance to the point where users can create perfect fakes, and it needn't be perfect to be believable in the right conditions anyway. Look at how much false or massively biased information is already flying around on the internet and just imagine how much worse it can get. I'm not saying this tech is bad, but there are too many negative use cases for it, and too many awful people in the world for this to not get used in the wrong way.

  25. This is REALLY f'g dangerous, imagine that now ANYONE has the possibility to reenact Trump & declare war at almost any country and it would ACTUALLY be believed!

  26. Why all the hand wringing and fear that this could be used for evil purposes. First it wouldn’t be hard to detect a fake video by analyzing the artifacts. Second, enforce watermarking or legal disclaimers… there are plenty of ways to make counterfeit money very well but it comes with such a big penalty that few dare take the risk. And 3rd, photos can already be altered so well you could hardly tell a difference by eye that it’s been doctored, have any untoward fake photos started the next world war? Any war?

  27. I'd love if you compared this approach to what's being done in the Deep Video Portraits paper:

  28. This is obviously technology designed to deceive and manipulate. Why would anybody want to defend such a thing? Now we cannot believe our own eyes and ears anymore, horrible people are taking that away from us. They steal our only ways to find out the truth and call it 'progress'. They treat our minds as playthings.

    Science is dead now, there is only technology left.

  29. Why did you not mention DeepFakes? It is much more capable in terms of degrees of movement and realistic looking, and should have been the state-of-art in this area. Plus, it has real practical use. wink

  30. Now we only need a neural network that takes a normal camera's image data and predicts the depth data of the image itself, and then pass the result to this technique.

Leave a Reply

Your email address will not be published. Required fields are marked *