Good callout re: tradeoff between control and verifiability.
The tough bit here seems to be that in the long term in order to build trust in the security of hardware signing the device would need remote attestation capabilities aka the ability to prove that the signing key is properly secured and that the right application code is being run (i.e. just sign raw sensor data and nothing else). These proofs would be useful in the context of ensuring that a device is always just signing raw images, but I could also see concerns arising about the device information revealed in proofs potentially being used for other filtering/gating. These types of concerns were at the root of the backlash from proponents of consumer device freedom and network endpoint agnosticism (i.e. servers shouldn't discriminate against what device an end user is using) against Google's now defunct Web Environment Integrity proposal (browsers would allow website devs to check what device the consumer is running). They're generally also at the root of concerns about the use of remote attestation capabilities in consumer devices generally.
But, at the moment, it doesn't seem that there is a way around this tradeoff...if the goal is to verify that an image was actually taken by a camera (although there could be workarounds if the goal is softened to something like establishing an image is *probably* taken by a camera based on information from your trusted circle).
C2PA and other such authentication schemes will need to preserve the location and time a photo is taken or else they will be subverted easily by simply taking a photo of a deepfake image (same with video) with a supported camera to add the authentication data. Even if the location is required, it would require the location and time reported by the camera cannot be spoofed, which could be done on a smartphone, but most standalone cameras like SLRs don’t have GPS or let the user add a location or set the time manually.
This is quite an elaborate solution in search of a problem. As we all know from the scientific literature, effective propaganda doesn’t need to rely on faked videos or photos; a fake headline will accomplish the purpose just as easily. We’ve had doctored photos for over a century, and high-quality photo doctoring was fully democratized over two decades ago by software tools like Adobe Photoshop. There is zero evidence that these digital tools have led to a measurable increase in the rate at which the public is led to believe false claims backed up by doctored photographic evidence. And there’s no reason to think that video won’t follow the same trajectory: perhaps we’ll have a few years in which a few rubes will be taken in who otherwise wouldn’t have been (just as a few of us may have fallen for photoshopped hoaxes in the late 90s), but then everyone will learn intuitively that video can be faked as easily as photos and we will settle into a new equilibrium that looks a lot like the present day.
I’m really getting quite tired of these evidence-free moral panics over AI. Perhaps it’s because I’m old enough to have lived through the last dozen of these: cryptography will enable child pornographers and terrorists to run wild, the web will lead to a post-truth era, Wikipedia can be edited by anyone so will be full of lies, DeCSS will lead to a world in which nobody makes movies, and on and on and on. And now, the AI doomers who at best whine about propaganda and bias (as though we’ve ever needed special software to perpetrate these evils) and at worst deploy the Condoleeza Rice gambit of “we don’t want the smoking gun to arrive in the form of a mushroom cloud”. The only constant is the hyperbolic, evidence-free assertions of certain doom and the collapse of civilization because of the latest digital tool. It really is tiresome.
I agree with you that the whole “social media deepfakes” panic is probably overblown.
However, what does strike me as a more realistic near-term concern on the affect of this technology on evidence admitted in court settings. I think we should stress test any technical standards against that use case, rather than trying to police all content put on the internet, which is a much more dubious pursuit in my view.
That makes perfect sense. It would be great if courts could rely on an authentication / chain of evidence system like this. Using a blockchain or other decentralized cryptographic authentication system, as suggested above, would be even better (probably). Anything that doesn’t require putting an expert witness on the stand would be better than the current system.
I used to work as a litigator, and while the evidentiary issues I usually encountered were pretty low-key, I did work in a field that asked for a lot of scientific and technical acumen from the court in order to understand the issues. Sadly, with a few exceptions, the bench, and court systems more generally, are some of the most conservative and technophobic institutions in North America. I’ve litigated in jurisdictions that still (as of five years ago at any rate) issued handwritten orders and that still didn’t have basic information (eg, hearing schedules) posted online. Firms would hire people to walk to the courthouse, photograph the hearing lists for the day, and email the photograph back to the office. No joke. (I won’t be coy: it was the Ontario Superior Court in Toronto.)
So as much as I’d like judges to start pulling up arrays of blockchain-authenticated evidence on their AR displays floating in the courtroom air like Tom Cruise in Minority Report, I’d settle for them learning how to answer an email. There’s a long list of reforms that would improve the efficiency of court procedure, and these reforms have been very unevenly adopted (some courts are actually pretty decent with this stuff).
I guess none of that is really a rebuttal to the sensible reform you’ve proposed. I’m just being a grumpy old man.
Good callout re: tradeoff between control and verifiability.
The tough bit here seems to be that in the long term in order to build trust in the security of hardware signing the device would need remote attestation capabilities aka the ability to prove that the signing key is properly secured and that the right application code is being run (i.e. just sign raw sensor data and nothing else). These proofs would be useful in the context of ensuring that a device is always just signing raw images, but I could also see concerns arising about the device information revealed in proofs potentially being used for other filtering/gating. These types of concerns were at the root of the backlash from proponents of consumer device freedom and network endpoint agnosticism (i.e. servers shouldn't discriminate against what device an end user is using) against Google's now defunct Web Environment Integrity proposal (browsers would allow website devs to check what device the consumer is running). They're generally also at the root of concerns about the use of remote attestation capabilities in consumer devices generally.
But, at the moment, it doesn't seem that there is a way around this tradeoff...if the goal is to verify that an image was actually taken by a camera (although there could be workarounds if the goal is softened to something like establishing an image is *probably* taken by a camera based on information from your trusted circle).
Almost seems like a reasonable use for a… blockchain? At least for high value human made assets.
Yes! I think so.
C2PA and other such authentication schemes will need to preserve the location and time a photo is taken or else they will be subverted easily by simply taking a photo of a deepfake image (same with video) with a supported camera to add the authentication data. Even if the location is required, it would require the location and time reported by the camera cannot be spoofed, which could be done on a smartphone, but most standalone cameras like SLRs don’t have GPS or let the user add a location or set the time manually.
Yup. There is no such thing as pure privacy and verifiably authentic media.
This is quite an elaborate solution in search of a problem. As we all know from the scientific literature, effective propaganda doesn’t need to rely on faked videos or photos; a fake headline will accomplish the purpose just as easily. We’ve had doctored photos for over a century, and high-quality photo doctoring was fully democratized over two decades ago by software tools like Adobe Photoshop. There is zero evidence that these digital tools have led to a measurable increase in the rate at which the public is led to believe false claims backed up by doctored photographic evidence. And there’s no reason to think that video won’t follow the same trajectory: perhaps we’ll have a few years in which a few rubes will be taken in who otherwise wouldn’t have been (just as a few of us may have fallen for photoshopped hoaxes in the late 90s), but then everyone will learn intuitively that video can be faked as easily as photos and we will settle into a new equilibrium that looks a lot like the present day.
I’m really getting quite tired of these evidence-free moral panics over AI. Perhaps it’s because I’m old enough to have lived through the last dozen of these: cryptography will enable child pornographers and terrorists to run wild, the web will lead to a post-truth era, Wikipedia can be edited by anyone so will be full of lies, DeCSS will lead to a world in which nobody makes movies, and on and on and on. And now, the AI doomers who at best whine about propaganda and bias (as though we’ve ever needed special software to perpetrate these evils) and at worst deploy the Condoleeza Rice gambit of “we don’t want the smoking gun to arrive in the form of a mushroom cloud”. The only constant is the hyperbolic, evidence-free assertions of certain doom and the collapse of civilization because of the latest digital tool. It really is tiresome.
I agree with you that the whole “social media deepfakes” panic is probably overblown.
However, what does strike me as a more realistic near-term concern on the affect of this technology on evidence admitted in court settings. I think we should stress test any technical standards against that use case, rather than trying to police all content put on the internet, which is a much more dubious pursuit in my view.
That makes perfect sense. It would be great if courts could rely on an authentication / chain of evidence system like this. Using a blockchain or other decentralized cryptographic authentication system, as suggested above, would be even better (probably). Anything that doesn’t require putting an expert witness on the stand would be better than the current system.
I used to work as a litigator, and while the evidentiary issues I usually encountered were pretty low-key, I did work in a field that asked for a lot of scientific and technical acumen from the court in order to understand the issues. Sadly, with a few exceptions, the bench, and court systems more generally, are some of the most conservative and technophobic institutions in North America. I’ve litigated in jurisdictions that still (as of five years ago at any rate) issued handwritten orders and that still didn’t have basic information (eg, hearing schedules) posted online. Firms would hire people to walk to the courthouse, photograph the hearing lists for the day, and email the photograph back to the office. No joke. (I won’t be coy: it was the Ontario Superior Court in Toronto.)
So as much as I’d like judges to start pulling up arrays of blockchain-authenticated evidence on their AR displays floating in the courtroom air like Tom Cruise in Minority Report, I’d settle for them learning how to answer an email. There’s a long list of reforms that would improve the efficiency of court procedure, and these reforms have been very unevenly adopted (some courts are actually pretty decent with this stuff).
I guess none of that is really a rebuttal to the sensible reform you’ve proposed. I’m just being a grumpy old man.