“And this is to say nothing of the fact that if we eliminate frontier open-source AI—setting aside what exactly what means or whether it is even possible—we will be guaranteeing that what I believe will be the most powerful technology ever developed is controlled exclusively by the largest corporations on Earth.” …. Perfectly said, it is so difficult to predict how all this will play out, but this portion seems extremely obvious at this point
I currently think that open source AI is slightly more dangerous than closed source (especially as AI models improve) but you make very smart points here. I’ll need to think about this more. Thank you for the thoughtful and deep post.
One other point I’d add is that for a determined nation state like China or Russia, all models are open source, since they can probably hack their way into a frontier AI lab and steal the weights. I
Thank you! Feel free to reach out to chat. Always happy to do so. And as I hope the piece conveys, I feel strongly about this yet have substantial uncertainty about whether things might change.
This was great. There seems to be an outdated perception that Chinese LLMs aren't competitive with US LLMs, and while that was true when LLaMA was released in Feb 2023 (which might be why it was used for this research), it's not true today, as you point out.
I also feel like the anti-open-source panic puts too much emphasis on weights and not enough on people and organizations. LLMs don't even last a year before they're replaced by better versions. Sure, the Llama 3.1 405B weights are valuable, but the Meta team that created those weights is far more valuable. Jealously guarding weights might confer some short-term relative advantage, but if it inhibits the domestic AI industry, that's a long-term loss.
“And this is to say nothing of the fact that if we eliminate frontier open-source AI—setting aside what exactly what means or whether it is even possible—we will be guaranteeing that what I believe will be the most powerful technology ever developed is controlled exclusively by the largest corporations on Earth.” …. Perfectly said, it is so difficult to predict how all this will play out, but this portion seems extremely obvious at this point
Thank you! I think centralization of power is still an underrated risk.
I currently think that open source AI is slightly more dangerous than closed source (especially as AI models improve) but you make very smart points here. I’ll need to think about this more. Thank you for the thoughtful and deep post.
One other point I’d add is that for a determined nation state like China or Russia, all models are open source, since they can probably hack their way into a frontier AI lab and steal the weights. I
Thank you! Feel free to reach out to chat. Always happy to do so. And as I hope the piece conveys, I feel strongly about this yet have substantial uncertainty about whether things might change.
This was great. There seems to be an outdated perception that Chinese LLMs aren't competitive with US LLMs, and while that was true when LLaMA was released in Feb 2023 (which might be why it was used for this research), it's not true today, as you point out.
I also feel like the anti-open-source panic puts too much emphasis on weights and not enough on people and organizations. LLMs don't even last a year before they're replaced by better versions. Sure, the Llama 3.1 405B weights are valuable, but the Meta team that created those weights is far more valuable. Jealously guarding weights might confer some short-term relative advantage, but if it inhibits the domestic AI industry, that's a long-term loss.
Thoroughly enjoyed this. Thank you.
Thank you for reading and for the kind words!