Unstable Diffusion

On (some of) the latest export controls

I am pleased to announce that I have launched a podcast called AI Summer with Tim Lee of the Substack Understanding AI; other subscription links here. Our first episode is with Jon Askonas from the Foundation for American Innovation. Much more to come. I hope you enjoy!

Also, for DC-area readers, come join AI Bloomers on January 21 at 5:30 PM. RSVP Here.

Introduction

Mechanized textile production was the high technology of the late 18^th and early 19^th centuries. Inventions like the spinning jenny and water frame—both pioneered in Britain—massively increased labor productivity. The deployment of these and related technologies offered an early glimpse at what industrial-scale manufacturing would mean for economics, politics, and labor relations. The world had not yet seen “factories” in anything like the modern sense, but these technologies presented the first opportunities for proto-industrialists to build the early iterations of what we now think of as the factory.

Naturally, Britain—the island empire, the world’s preeminent naval power—was home to much of this early innovation, just as it was the cradle of the first industrial revolution. And the British guarded this technology jealously. Parliament passed increasingly stringent restrictions on the export of tools used in textile manufacturing, designs for the most innovative equipment, and the manufacturing equipment itself throughout the final decades of the 18^th century. They forbade workers with specialized knowledge of textile manufacturing from emigrating, and they prohibited foreigners from entering the early textile factories. They strove vigorously to maintain British leadership in textile manufacturing.

But there was a hungry upstart entering the picture: the United States of America. What began during this era of British technology policy as a British colony ended as an independent country, victorious in war against the empire. We Americans had no interest in maintaining leadership. We had no interest in protecting our technology or preserving our competitive edge. We wanted, simply, to compete, to build and, perhaps one day, to lead ourselves.

Industrial machinery was easy enough for the British to hoard within their borders, but the knowledge inside a person’s mind was far harder to contain, even in an era whose information and communication technologies were far more limited than what we have today.

Eventually, an enterprising Brit by the name of Samuel Slater, who had served as an apprentice to the inventor of the water frame, made his way to America. There he collaborated with New England financiers to create an American textile manufacturing venture. Around the same time, an American named Francis Cabot Lowell managed to make it into British textile factories, memorizing the design of their equipment. His transplantation of those ideas—and not insignificant innovations of his own—gave rise to the Lowell System, a template for American industrial production that would be replicated many times over. Lowell, Massachusetts, bearing his name ever since, would become the birthplace of American capitalistic dynamism.

Within a few decades, American textile manufacturing would equal, if not surpass, British output. And we sold our textile manufacturing equipment internationally, too. At first, perhaps our equipment was not as good. But that mattered little, because the world’s best machines—the ones made by the empire on the island—were illegal or difficult for other countries to buy. And eventually, by virtue of making the machines at great scale, America became better at that, too, than the British.

I often whether the British could have played their cards differently. I wonder what the world would have looked like if the British had supplied America—and everyone else on Earth—with textile manufacturing equipment, rather than keeping it mostly on their island. But I also wonder whether the British really had a choice. Textile manufacturing was perceived as a key domestic competitive advantage, as a pillar of Britain’s economic and even military dominance. This was a matter of national security. What else were they to do but protect?

Surrender, disguised as protection and nationalism. This, for the British, is how it happened. History can be cruel.

One of the lessons I take from history is that a great power sometimes has no choice but to walk a certain path. Sometimes, you are simply making the best wagers you can with the cards you have in your hand.

The AI Export Controls

I have not commented much about America’s ever-escalating export controls on AI computing hardware and the semiconductor manufacturing equipment used to make that hardware. One of the simpler reasons for this is that I believe my opinions on the matter will please no one.

I have, in private and occasionally in public, described the 2022 decision to ramp up export controls as either the wisest or the most disastrous decision of President Biden’s administration. There is some chance that they will be perceived by historians as a forward-looking policy move (remember, they began before ChatGPT!) that set the stage for American dominance in AI during the crucial years leading to “AGI,” “transformative AI,” “powerful AI,” or whatever moniker you prefer. There is also a chance that they will be perceived a bit like the British decision to hoard textile manufacturing equipment: a surrender on the field of economic competition, a preemptive ceding of victory to our adversary, an attempt to protect rather than to win.

I do not know what the final judgment on the export controls will be, but more importantly, and just like the British, I am not so sure we have a choice. Even with the export controls, America’s top AI companies and hyperscalers are compute-limited. Imagine a world in which the United States had no export controls on the high-end GPUs needed to train the best AI models, and that a large share of Nvidia’s H100s had been sold to Chinese companies over the past 18 months.

“We could train larger models, or we could offer our existing models at a lower price,” one can imagine Sam Altman or Dario Amodei or Satya Nadella saying to a reporter or at an event, “were it not for those million chips Nvidia sold to China.” Would American policymakers really tolerate such a reality for very long? I doubt it.

Counterfactuals aside, we especially do not have a choice now that we have started down this path. Ever since we first played this card (which, to be clear, was during the Trump administration in 2018 and 2019, though these were far more limited, targeted only at specific firms: ZTE and Huawei), the Chinese have doubled down on developing a fully domestic semiconductor manufacturing industry. They will not stop simply because we begin selling them more GPUs or semiconductor manufacturing equipment; if anything, such a loosening of the export controls would only help them.

So we are left, really, with just one option: to play our hand as best we can.

Understanding the Diffusion Rule

America’s first major effort at AI-related export controls came in October 2022, and in the ensuing months it became obvious that they failed to meet their goal. One of the key metrics used in those controls was interconnect bandwidth, or the speed at which data is transferred between chips. Thus, Nvidia could ship a chip to China with the same computational performance as the flagship A100 and H100 AI GPUs, so long as the interconnect bandwidth was lower. These chips were called the A800 and H800, and Chinese firms bought them in droves.

The 2023 iteration of the export controls plugged that “loophole” (I dislike this term, and find it unfair to demonize Nvidia for selling a chip that complied with the law, even if the law in this case was sloppily drafted). But there were a wide variety of other enforcement problems, and simultaneously many firms lobbying the American government to adjust the controls in this or that way. The result has been hundreds of pages of dense and highly technical rules issued over the past two years. The 2024 iteration alone (in two parts) is over 200 pages. This is the inevitable result of technocratic governance: rules beget more rules, and complexity begets more complexity.

Unaddressed by these gargantuan sets of rules, however, was the fact that Chinese companies could route their GPU orders through countries like Singapore, or even use cloud computing services from unrestricted companies throughout Southeast Asia.

The new “diffusion rule,” hinted at in various ways for months and formally introduced earlier this week, aims to correct this and other perceived problems by, in essence, ensuring that the federal government manages the worldwide rollout of AI. This may sound hyperbolic, but don’t take it from me. The stated purpose of the rule is to “regulate the global diffusion of the most advanced artificial intelligence models and large clusters of advanced computing integrated circuits.”

The diffusion rule does this through another 168 pages of rules, which do not replace but exist alongside the 2024 export controls. The rule is not just about identifying new products over which to exercise export controls, but also about building a broad framework under which to conduct the controls. Keep in mind that not all chips designed or otherwise produced by American firms are controlled by this framework; indeed, the vast majority are not. The diffusion rule only applies to the highest-end chips necessary for training and running large AI models.

The framework operates by dividing all countries on Earth into three tiers: one consisting of the US, other Anglo countries, most of Western and Northern Europe, Korea, Japan, and Taiwan (tier 1); one consisting of enemies such as China, Russia, and Iran (tier 3); and the third consisting of the rest of the world (tier 2). Countries in the first group, which for some reason excludes some NATO members as well as Israel, get mostly unrestricted access to chips; countries in the “enemy” group get no export controlled chips (which was true before this rule); countries in the “most of the people on the planet” group get some chips, with significant obstacles and strings attached.

Specifically, any individual company in most of the countries on Earth can import a little fewer than 27 million units of “total processing performance” (TPP) apiece per year. TPP is a metric calculated by multiplying the theoretical peak number of the operations per second for multiply-add calculations (W=(X*Y)+Z) by the bit length of the operation and then doubling that.

This translates to about 1,700 Nvidia H100s or “H100 equivalents” per year. But the H100 is soon to be a previous-generation chip, now being replaced by the much-faster B100. I did not calculate the TPP for the B100, but given its performance improvements over the H100, I’d guess it translates to around 340-850 B100s per company per year. Nvidia releases new top-end AI chips every year, so it would not be unreasonable to expect the number of actual GPUs allowed under this cap to drop dramatically over time.

Every individual order above 1,700 H100-equivalent GPUs requires an export license and counts toward a country’s cap of 50,000 H100-equivalent GPUs per year. This export license needs to be reviewed, on a per-order basis, by four US government entities: the Department of Commerce, the Department of Energy, the Department of State, and the Department of Defense (they vote to make a final decision; I do not entirely understand what happens if they tie. UPDATE: Apparently, if they tie, the license is rejected. Thanks to Charles Yang for letting me know!).

If this sounds tedious to you, you are not alone. The federal government also finds this tedious, and they have struggled with approving licenses for large orders in countries we consider “semi-friendly,” like the United Arab Emirates. So the diffusion rule creates a “national validated end-user” (NVEU) status for which individual companies in tier 2 countries can apply.

With NVEU status, an individual company can receive up to 320,000 H100-equivalent chips between now and the end of 2027. To receive this status, the company must adhere to certain use restrictions and (significant) cybersecurity standards; my expectation, and the expectation of any rational company spending billions of dollars on GPUs, would be that these requirements will become more stringent over time. After all, rules beget rules.

In addition to this, however, any company in a tier 1 country (the “friend” group, and obviously, the US) can receive a universal validated end-user (UVEU) status, after which they can deploy chips around the world with somewhat simpler restrictions. UVEU companies can deploy as much as they want, so long as no single tier 2 country receives more than 7% of their total worldwide compute capacity, and so long as 75% of their total compute is deployed within tier 1 countries. For US holders of UVEU status (and here, think: essentially every holder of UVEU status), there is an additional requirement that 50% of total compute capacity be deployed on American soil.

Finally, the diffusion rule requires a license for any “export” of US model weights that required more than 10^26 training FLOP, including fine-tuning/reinforcement learning but excluding synthetic data generation. There is a crucial exemption for open-weight models, meaning that this rule only applies when a hyperscaler is seeking to use a closed model in a data center located in a non-US country. Whenever this occurs, receiving approval for a license will require adherence to many of the same stringent cybersecurity standards mentioned above in the context of attaining an NVEU status. Like NVEU holders, these data center operators will be required to surveil their customers for potential violations of the diffusion rule’s use-based restrictions.

Given the number of UVEU (read: US) companies that will be deploying advanced model weights in tier 1 countries, I suspect that over time, the cybersecurity standards in this “export” rule will become de facto standards for AI data centers in the US. On top of all the other regulations, then, the diffusion rule is also probably a backdoor to domestic policymaking.

The compute-based threshold that triggers export controls for model weights rises, basically, when a new open-weight model exceeds the threshold. So, if an open-weight model is released that required, say, 3 x 10^26 flops to train, that will become the new threshold for triggering export controls on closed-weight models.

From what I understand, no tier 2 country is permitted to host these “advanced” model weights on local data centers. Indeed, no tier 2 country, practically speaking, can train frontier models under this rule.

That’s an awful lot of authority for the American government to assert, with precisely no Congressional input, over hardware that, by and large, is not manufactured in the United States. I suppose you could call that a well-played hand.

Conclusion

Is this rule better than what came before? Sure, in the sense that it plugs enforcement gaps created by the old rules. But fixing problems with the previous rules means that those problems originated, by definition, from the existence of those rules in the first place. And the earlier rules fixed problems with even earlier rules, which in turn were created by those rules. None of these “problems” would have ever existed had we not started down this path of attempting to regulate the global use of high-end computers. But we did, and as I’ve said already, I’m not sure we could have avoided this path.

Given that we really do appear to be on a short-term trajectory to very powerful AI systems, I understand the logic in seeking to exercise control of some kind over the global compute ecosystem over the next few years. Yet at the same time, we are expending a lot of political capital—and creating many burdensome rules—to secure an outcome that probably would have happened anyway.

That’s the irony: Despite being an exceptionally broad assertion of American power, the diffusion rule does not change much about the status quo. American firms already dominate the global buildout of AI infrastructure. Most of the AI compute deployed in the world is already in the United States. Very few countries are attempting to acquire hundreds of thousands of H100s, and if anything, the diffusion rule is likely to simplify their efforts to do so, if only because companies in those countries will be able to proceed with one license, rather than negotiating ad hoc deals.

Indeed, one way to think of the diffusion rule is locking in the status quo of AI compute deployment, which favors America. It’s just that, instead of locking it in through winning in competitive markets, we’re doing it through rules—rules that make it convenient not for American companies, but for the American government.

And it locks in the status quo in more ways than one. This rule is based on many assumptions about AI itself—that high-end GPUs will remain the predominant form of compute for AI and that they will remain scarce (looking at you, Beff); that decentralized training will never work at large scale; that making models bigger will always be the primary of making them better. Innovations along these vectors would be inconveniences for our regulators, and perhaps will be perceived by them in an adversarial manner. Viewed in this light, innovations become a threat to power, not a benefit to growth. This dynamic is one manifestation of the oft-discussed “regulatory capture,” and it far from the only manifestation these rules could foreseeably create.

Remember that this set of laws originated out of the Biden administration’s “small yard, high fence” mantra, or the idea that we want to prevent a limited number of GPUs from being used in Chinese military applications. But then we began to worry about whether Chinese companies were using data centers in Southeast Asia. And we wondered about whether these ambitious new AI initiatives in the UAE and other Gulf states were, just maybe, connected to the Chinese in some way—or whether they could be some day. And then there was the smuggling of chips into China. And what about model weight security? And what about whatever comes next? Did you know none of this covers very powerful inference chips—the kind used to run models like OpenAI’s o1 (or Chinese lab DeepSeek’s r1) for long periods of time to extract better performance? We’ll have to do something about that, won’t we?

We have gone from “small yard, high fence” to “regulating the global diffusion of advanced AI” in a little less than 30 months. You have just watched a regulatory regime grow by the better part of a thousand pages in three years, without a single member of Congress voting for anything. This is what technocratic governance looks like. Rules beget rules. They almost have a mind of their own, like self-replicating automata. This kind of spiraling regulation is the sort of thing classical liberals like me warn about every time we embark on some new regulatory adventure. Of course it ended up like this.

The Trump administration can choose to walk at least some of this back, but it is not obvious to me how. Sure, they can eliminate the restrictions on model weights; that would be an easy win. They can walk back some of the cybersecurity requirements and use-based restrictions. But at its core, the diffusion rule really is fixing problems (“loopholes”) in the old rules, and there are many ways in which this rule does advantage American companies (especially our hyperscalers, now undeniably in pole position to lead AI infrastructure development all over the world). I suspect any real improvement would have to come from a fundamentally different approach.

Absent those novel ideas, though, we are left playing the cards we have. Our perpetually underfunded Bureau of Industry and Security (the part of the Department of Commerce tasked with enforcing most of the diffusion rule) will have to figure out how to put all this into practice. I fear they will struggle. So, too, I suspect, will the companies regulated by this document.

With so many rules, one thing is nearly certain: Sooner rather than later, we will need yet more rules to fix the problems with these rules. I am not sure we have much of a choice. Perhaps this, for America, is just how it will happen.

But maybe there is a much better hand to be played. If someone has fresh ideas, I’m all ears. I hope the Trump administration is, too.