Deepfakes - The Peril and Potential
Added 2024-02-15 17:20:08 +0000 UTCTake a 14 minute tour with me of the cutting edge of deepfakes, from speech-to-speech to politics, YouTube and business. We'll discuss the upsides, including with a senior figure at Elevenlabs - and you'll get to hear my voice with a different content and personality - business potential, as well as threats and downsides. Including developments from just the last week, new papers, an exclusive interview and more, this is the <15 min primer on all things deepfake.
Elevenlabs - https://elevenlabs.io/speech-synthesis?task=speech-to-speech
Slovakian Election - https://edition.cnn.com/2024/02/01/politics/election-deepfake-threats-invs/index.html?s=09
Superbowl Rewritten - https://twitter.com/BecketAdams/status/1757470253930430531
Mayor of London - https://www.bbc.co.uk/news/uk-68146053
Heygen - https://app.heygen.com/home
AI voice returns - https://www.bbc.co.uk/news/technology-44481510
Seinfeld - https://www.youtube.com/watch?v=S1MBVXkQbWU&t=90s
Handwriting - https://www.bloomberg.com/news/articles/2024-01-15/after-voice-clones-and-deepfake-videos-ai-can-now-mimic-handwriting
New Laws - https://www.nbcnews.com/tech/tech-news/deepfake-bill-open-door-victims-sue-creators-rcna136434
https://refuge.org.uk/news/new-laws-criminalise-the-sharing-of-intimate-deepfakes-without-consent/
2.2% Deepfake Victim Rate Paper - https://arxiv.org/pdf/2402.01721.pdf
Comments
I think we need to start discussing that deepfaked porn is going to be a heavily gendered problem. Yes, it's going to affect men as well, but I think it is quite likely that MOST women now under 16 will be confronted with being deepfaked while at school; and basically ALL women choosing to play a prominent role in any public capacity will have this happen to them, with added emphasis on any woman running for public office or engaging in political speech. We don't just need to declare it illegal, we need to give the right resources to law enforcement to go after this hard. And we need to start doing that now. Once open source tools become a bit better and bit more easy to use, this is going to explode. We should be acting by then.
Jörg Weiß
2024-04-27 13:05:17 +0000 UTCWow... I'm less than 5 minutes in and I'm already blown away
Joshua Davis
2024-02-26 15:09:01 +0000 UTCOne thing that you might have missed: Plausible Deniabilty for anyone! -> Increasing prevelance of 100% believable synthetic media does have the side effect of eroding trust in ANY media... Whenever stuff like Trumps "just grab em by the p****" pops up again... The argument will be "AI Fake". Ironically enabling trumps calls for "FAKE NEWS, FAKE NEWS" even more. This will add an absolutely increased burden on reliable news media. If everything can be faked... nothing again will be able to be claimed true with 100% certainty. With the polarization going on... valid proof of authenticity doesnt matter to opposing party. I only see broad and wide AI literacy education, legislation and platform moderation together to give us a fighting chance. Otherwise we are pretty doomed tbh.
Stefan
2024-02-24 20:47:46 +0000 UTC100% on board with the considerable lack of AI literacy and Awareness as well as the undercommunicated downsides of opensource. Its great and all that the closed sourceplatforms are looking into a few things to prevent misuse. However... the main bad actors are probably gonna go opensource anyways. And any everyday joe will be encouraged to look into any of the many "make a quick buck" 1 click deep fake apps or just go straight open source. In this space I mostly see downsides with very weak upsides. Genie is out of the bottle though.
Stefan
2024-02-24 20:43:38 +0000 UTCArg! What a shame
Erik
2024-02-19 09:11:58 +0000 UTCI’d say the lack of English accent was the most disqualifying factor, if the point would have been to make it sound authentic. And of course the whole message being an antithesis to Mark’s business world view. So… I guess my internal neutral network employed a richer model of the person, judging the authenticity of the video by how well it matches my own predictions of what that person would sound like in German, given they are a non-native speaker and what kind of message would be plausible coming from them.
Pavol Vaskovic
2024-02-19 07:31:17 +0000 UTCYes, the blind trust in media will be shattered but I'm not convince it is a bad thing. I grow up in the end of comunist Poland. You could listen to the news from oficial propaganda chenels or from Free Radio sponsored by CIA. I think more people where critically thinking back then. Also for 99% of human history the news was a gossip, and all pictures where painted. I think we warry too much about some end of XX century anomaly disappearing.
Arek Stryjski
2024-02-16 22:05:12 +0000 UTCYuck. WorldCoin is blockchain based, and fewer and fewer people are taking blockchain seriously. (Plus the name "WorldCoin" is dystopian...). See my long reply to Shawn Fumo's comment on my comment... we don't need any new decentralized blockchain magic to make this work, we just need to leverage existing technology and real-world trust thoughtfully and systematically. As Shawn rightly pointed out the hardest part will be enough people coordinating on this to achieve critical mass.
James Kittock
2024-02-16 14:33:53 +0000 UTC"Most people just aren't aware of how far things have come" Yup, this is really the biggest problem. We need coordinated education, much as has been done for other societal issues in the past (for example, when I was a teenager the US was pushing hard on "drunk driving is bad"). The problem with that is our government officials are old and crusty an don't understand either. They think they can solve through restriction rather than through education. I am less worried about the banking deep fake thing, since I think a few highly publicized cases like that will both serve as a visible warning *and* will cause institutions to up their security protocols. This has been happening slowly over the past decade or so (2FA, security keys, etc.) but I think it's going to be catapulted forward by the need to protect against a new kind of spoofing attack.
James Kittock
2024-02-16 14:30:42 +0000 UTChaha fair
James Kittock
2024-02-16 14:26:03 +0000 UTCi speak very strong german (austrian dialect) and in my region people speak quite differently just because they grew up in the other valley on the other side of the mountain. do you think that faking all these variations of dialects and accents could be a problem. there is not really a lot of traning data for all these variations (i think) ?! what do you think?
Christopher Pollin
2024-02-16 14:14:49 +0000 UTCI relate to Elon Musk when he talked about the feeling of being one of the few people on a spire far above the rest when it comes to seeing how things will play out. When chatgpt became popular I said the internet was fucked. I knew immediately that the internet would die and become a wasteland of untrustable material. Could bring about the mark of the beast, a method of checking for reality as opposed to robot. Also I was watching the video on my phone and when you played your AI crypto clip I couldn't hear a difference other than maybe the speed of your talking. I had to replay it because I missed it the first time.
Jonathan Kirk
2024-02-16 14:05:10 +0000 UTCAll good points. I am not convinced that device provenance is that useful. I am thinking more of creator & subject provenance. For example, I was just looking at a photo of Biden that said "Saul Loeb/AFP/Getty Images". The problem is, anyone could fake provenance of an arbitrary photo just by putting that string under it. But if the photo was e-signed by Saul Loeb, the AFP, and Getty Images, then it would be clear that the provenance was legitimate. Similarly, I could e-sign photos of myself and only those photos would be considered legitimate. You are right about chain of modifications being a challenge, however I think that is also addressable via nested e-signatures. Take the case of the photo of Biden above. If I take a screenshot, there will be no provenance information. If I download it, it will be e-signed, but to modify it I will have to "open" the envelope and won't be able to save a modified version with the original signature. I could save my version with my own signature, but it wouldn't have the original provenance information. Alternatively, some relatively simple tooling would allow me to save and sign my version along with the original untampered version. So my e-signed envelope would contain my version plus the e-signed envelope for the original. All of this of course leads to the natural question of root of trust. Just because the photo was signed by someone named "Saul Loeb", how do I know it's the *real* Saul Loeb? At least in the near term, I think the only plausible answer is that widely trusted entities provide root of trust, and they transfer this trust to other entities. This is similar to how DNS works: the IANA is the root of trust, and then trust spreads out eventually ready service providers such as GoDaddy. [Fun fact: the root of trust of the DNS system is a sheet of paper signed by a bunch of people in a public ceremony. You can learn about it here: https://www.iana.org/dnssec ... I wonder if this process will become vulnerable to deepfakes!] For individuals, the obvious root of trust will be governments, which provide "real world" identity in the form of passports and other official IDs. For organizations, it may be a bit trickier since there are a wide variety of registration schemes across jurisdictions, shell companies, DBA's (fake business names), etc. In the near term, the most pragmatic solution is to link organizational identity with web addresses. So if something is signed with a private key that has its public counterpart on whitehouse.gov, we can assume it is an official Whitehouse photo or document. Still have to worry about private key theft and fake private key injection, but it gives a starting point. I think you are spot on about coordination being the biggest hurdle. I doubt any new technology is needed. What we need are the standards and workflows that integrate verifiable identity/provenance into the fabric of the digital realm. I personally think this is perhaps the biggest societal problem we will face over the next few years, so I hope governments and tech players take it seriously. Finally, I am skeptical about the idea of "declared fake" content. It will always be possible to work around, and compliance will be effectively impossible to guarantee. It makes much more sense to identify the real stuff and teach people that unverified stuff should be assumed to be fake.
James Kittock
2024-02-16 13:50:17 +0000 UTCYeah, though provenance has its own issues. Like if a video can be traced back to a particular camera or if the GPS location of it is part of the signature, that has privacy implications. And then issues of how much editing of various kinds is ok. You can imagine a chain of things where the original video signed by a camera and Premiere Pro keeps track of every cut made, with links to the hashes of the original videos with the original time codes. But it starts to be a lot of data to keep track of. For a photo, does Lightroom need to store what contrast settings were used and what acne removed? And does it encroach too much on artistic secrets by having the “source code” of all modifications available to the viewer? Maybe we don’t use the chain of mods and people just sign the last version they approve of and keep the original vids around for legal cases but that doesn’t help as much for disinformation that doesn’t purport to be from an official source. It all feels like a lot of coordination needed from many different players from cameras, to editing software, to social media sites. But I guess whatever things we can do to push incentives toward the right thing are still helpful. If the major AI image generators give watermarks that stay even if resized or cropped a little, and social media sites display that, it is a start. Plenty of people aren’t trying to deceive with AI images but people end up sharing them and others assume they may be real. So that helps that case even though there’ll be plenty of ways to generate images without watermarks (like with open source models)
Shawn Fumo
2024-02-16 13:41:59 +0000 UTCAs a native German speaker, I can say that the pronunciation is almost perfect. It's very easy to understand and accurate. It does sound a bit robotic though, which, in Mark's case might add to its authenticity? ;)
Daniel Schönbohm
2024-02-16 11:46:09 +0000 UTCWasn't allowed to video it!
Philip
2024-02-16 10:31:16 +0000 UTCThis is why Altman is on about WorldCoin but hopefully easier and more palatable solutions arise.
Philip
2024-02-16 10:30:56 +0000 UTCIm Shared what will come. But Great Video thanks
Rico Rauschkolb
2024-02-16 09:32:04 +0000 UTCInteresting timing with this coming out the same day OpenAI unveils Sora. :) I'll start by noting that this is not something entirely new: it's been happening as long as technology has been evolving. For example, consider Orson Welles' "War of the Worlds" incident. Or consider this example from the early web: https://www.snopes.com/fact-check/carey-on-starving/. And, of course, anyone with strong Photoshop skills had the ability to make believable fake images for the past 20 years or so. Having said that there are some dynamics that make this situation unprecedented. 1. Technology is evolving faster than most people can follow. This means that most people lack a frame of reference for making judgments about what is real and what is fake. 2. However, younger generations are growing up with these technologies and will be much more savvy. 3. While fakes have been possible for a long time, they are now being "democratized" so that pretty much anyone can do them without specialized knowledge or massive expenditure. 4. There is not yet consensus on the appropriate way to combat fakes. The answer probably lies with some mechanism to prove provenance cryptographically, but we are far from anything workable solutions, much less an industry standard. Given all of the above, I think we are in for a year or so of massive growing pains as large swathes of society are taken in by deepfakes, followed by some amount of time where everyone is highly skeptical. This could lead to a pretty massive shift in how consumers engage with content. Ultimately I do think we will collectively settle on an authenticity proofing mechanism that will allow people to once again trust some content. But there's going to be chaos between here and there most likely. BTW, regarding STS, keep an eye on Aloud by Google/YouTube: https://www.theverge.com/2023/6/22/23769881/youtube-ai-dubbing-aloud
James Kittock
2024-02-16 04:09:08 +0000 UTCAny ideas about how to defend against deep fakes? I was thinking on something that uniquely identify you as human. Biomarkers could work or something simpler like a 2FA. Anytime my partner asks me for money, I should ask her for the 2FA authenticator code.
Carlos Baraza
2024-02-16 00:09:47 +0000 UTCI have a friend doing his PhD on how people are manipulated by social media algorithms/content and what that means for our society. We've been talking about these points for the last few months - AI brings new challenges that are coming very soon, and however bad we think social media is now, it's about to get crazy, possibly dangerous. Are you going to upload this one to YouTube to be shared?
Tim Hennessy
2024-02-15 23:54:49 +0000 UTCWe're already in pretty crazy territory and don't see how it'll get better instead of worse. Most people just aren't aware of how far things have come, or know how to access the tools. This is one of the areas where I feel most nervous about the open source side of things. Right now anyone for free and without an internet connection (after downloads of course), could make deep fakes on their phone. Any model that supports in-painting and knows about nudity basically enables it by definition. And feels hard to crack down in totality on that when there are so many legitimate uses and when the files in question aren't that large (2gb for SD1.5 models). I've seen AI video creators use face swap to help consistency/quality of characters and lip-sync tools like heygen to get them to say things in the video. But as this video points out, can also be used for any other purposes. Some laws prosecuting offenders seems a good start, but I have my doubts on how much it'll really change things. Sure coordinated robocalls will probably be prosecuted, but will a random video that appears on TikTok from an anonymous user? That banking deep fake thing is also really scary. Even if it was quick, the fact that they could convince an employee it was a real conference call is crazy. I saw an SFX channel did a live deepfake thing last year that was starting to get there, so probably is even farther now. Think about any business you are in where if someone breaks into an account on Teams, they've already been able to cause big problems like asking for a password. Now imagine it is a video call. That is how you would have detected it was fake before, but now you don't even have that to go on.
Shawn Fumo
2024-02-15 21:44:45 +0000 UTCGreat video. Are you releasing your interview with Suno separately Philip?
Erik
2024-02-15 20:55:37 +0000 UTCIt's already the wild west ... it's going to become the wild west amplified a million fold!
Kelvin Parker
2024-02-15 19:42:13 +0000 UTCI see a lot of potential for businesses to easily create training videos for staff. Same thing for company podcasts or anything similar. Either way this stuff is getting exciting! Not to mention OpenAI’s new text to video model
Trenton Dambrowitz
2024-02-15 19:13:09 +0000 UTCGlobal Lingua Franca incoming! This is going to structurally reshape the sharing of narratives. But as with all technologies, it is dual use. This is a fundamental principle - all technology is dual use. The ability to universally translate is commensurate with the ability to do deepfakes and misinformation.
David Shapiro
2024-02-15 18:00:40 +0000 UTCDeepfakes scare me to death, more than any other AI tech existing right now. There are just too many players profiting from chaos and confusion.
Phillip Yao-Lakaschus
2024-02-15 17:46:46 +0000 UTCThank you, Philp. I'd say you're pretty much spot on with your take on Deep Fakes.
Tony Coffman
2024-02-15 17:39:48 +0000 UTC
