Interesting proposal. I think it ends up becoming something like the equivalent of insurance against a future tort liability, which definitely has useful features and also overhead. That might even be a more direct method. The thing that I have the biggest question with is that none of the AI safety research or assessments I have seen ar…
Interesting proposal. I think it ends up becoming something like the equivalent of insurance against a future tort liability, which definitely has useful features and also overhead. That might even be a more direct method. The thing that I have the biggest question with is that none of the AI safety research or assessments I have seen are nearly good enough to be enough for a certification. Definitely not enough to have any views on whether it could cause particular forms of mayhem.
Agreed re: insurance. But the problem with insurance is straightforward: no insurer really wants to touch this space, because no one can quantitatively model the risks. To do that, we need real-world deployment data--lots of it. And to do that, we need to accelerate real-world deployment, which this proposal is intended to do. Ultimately, I expect that optimal governance mechanisms would converge around a price system of some kind--either insurance literally or insurance by another name. But you cannot will a market into existence when the market does not "want" to exist, which is where we currently stand.
I think that view of AI safety is still accurate, though much less accurate than it would have been a year ago. There really have been appreciable advancements, and I anticipate this to continue such that, in a year or so (the earliest possible time this system could be put into place), things will be in a more mature state. It'll still be evolving, though, because the technology moves quickly.
I'm not arguing for insurance, just noting that the fact that nobody can price the risks well enough to provide insurance is a great case and point of how we are basically dealing with unknown unknowns at best. Certification agencies will have the exact same problem.
And agreed we need more deployment data. That'll happen in due course.
True, most standards today only support process audits. We are likely some time away from establishing concrete, measurable technical standards that would enable certification testing, especially across safety layers (e.g. input/output classifiers, feature clamping, etc.). Developing such standards would require standard-setting bodies to have an agility and coordination with frontier labs that just does not seem to be present today.
Interesting proposal. I think it ends up becoming something like the equivalent of insurance against a future tort liability, which definitely has useful features and also overhead. That might even be a more direct method. The thing that I have the biggest question with is that none of the AI safety research or assessments I have seen are nearly good enough to be enough for a certification. Definitely not enough to have any views on whether it could cause particular forms of mayhem.
Agreed re: insurance. But the problem with insurance is straightforward: no insurer really wants to touch this space, because no one can quantitatively model the risks. To do that, we need real-world deployment data--lots of it. And to do that, we need to accelerate real-world deployment, which this proposal is intended to do. Ultimately, I expect that optimal governance mechanisms would converge around a price system of some kind--either insurance literally or insurance by another name. But you cannot will a market into existence when the market does not "want" to exist, which is where we currently stand.
I think that view of AI safety is still accurate, though much less accurate than it would have been a year ago. There really have been appreciable advancements, and I anticipate this to continue such that, in a year or so (the earliest possible time this system could be put into place), things will be in a more mature state. It'll still be evolving, though, because the technology moves quickly.
I'm not arguing for insurance, just noting that the fact that nobody can price the risks well enough to provide insurance is a great case and point of how we are basically dealing with unknown unknowns at best. Certification agencies will have the exact same problem.
And agreed we need more deployment data. That'll happen in due course.
True, most standards today only support process audits. We are likely some time away from establishing concrete, measurable technical standards that would enable certification testing, especially across safety layers (e.g. input/output classifiers, feature clamping, etc.). Developing such standards would require standard-setting bodies to have an agility and coordination with frontier labs that just does not seem to be present today.