The OpenAI Pastiche Edition

Musings on the latest drama, extra thoughts on the o1 models

I want to experiment with a new format this week. Rather than one long essay, I’m going to share some quick takes and additional thoughts on various recent happenings. This week, all happen to relate to OpenAI.

I’ve been meaning to try something like this for a while, and to be perfectly frank, this week is a good time to try: I am currently knee deep in a monster of an essay. It’s my most ambitious project thus far, and I look forward to sharing it with you soon. In the meantime, let’s dive in.

In other news from me: I wrote a brief public interest comment in response to the AI Safety Institute’s proposed guidelines for mitigating the misuse of foundation models. The tl;dr is that these guidelines almost entirely ignore the existence of open-source AI, which makes them both problematic and less useful than they otherwise could be. Also, I appeared in the inaugural episode of Virginia Postrel’s new podcast. It was a fun conversation. And I had my first report published by Mercatus—it is about the need to think of the “AI workforce” more broadly. Several more Mercatus reports are in the hopper, and these are in addition to the major project I mentioned above.

For DC-area readers: one week from today, I am co-hosting “AI Bloomers,” a get together for people who are excited about AI. It will be at Union Pub on Capitol Hill from 6:00-8:00 on Thursday, October 3. RSVP here.

Thoughts on the OpenAI departures and restructuring

The OpenAI drama this week is twofold: first, several key OpenAI executives—Chief Technology Officer Mira Murati, VP of Post-Training Research Barret Zoph, and Chief Research Officer Bob McGrew. Second, multiple outlets are reporting that OpenAI intends to restructure itself as a for-profit Public Benefit Corporation, with CEO Sam Altman taking an equity stake in the new company (up until now, Altman had not taken any equity in OpenAI).

Every time there is OpenAI-related drama, the safety community cynically seizes the narrative. The decision to re-structure as a Public Benefit Corporation is, we are told, the “mask coming off,” even though the PBC is precisely the same structure that OpenAI rival Anthropic employs. “This is the strongest closing argument I can imagine on the need for [SB 1047],” wrote Zvi Mowshowitz. While Zvi does not directly imply that the departures are related to the restructuring, others do; the general sense I get from scanning X is that surely, these executives left because of internal struggles related to the restructuring.

I am not so sure. OpenAI Kremlinology is a favorite pastime of the AI community. As a nerd with more than two decades of armchair Apple Kremlinology experience (I remember when Gurman was a kid working at 9to5Mac, outclassing the big boys at the Wall Street Journal and Bloomberg from his dorm room!), I’m here to tell you: not everything that happens inside a company is fodder for the narrative you’re pushing.

If you believe that we need SB 1047 because of the imminent rise of a species of superintelligent machines with intent to kill humanity, it seems to me that there are, in fact, stronger closing arguments on the need for that bill than the departure of some executives and a corporate restructuring.

Let’s start with the restructuring. I fail to understand how OpenAI adopting the same corporate structure as Elon Musk’s xAI and Anthropic is a “mask off” moment. Are those companies somehow less safety-conscious than OpenAI? If anything, the evidence points the other way: Elon Musk, famously, supports SB 1047, and Anthropic has made AI safety one of its key research and marketing differentiators.

I, along with most observers I know, expected OpenAI to pursue a restructuring like this at some point. It is unclear that it means much of anything for OpenAI’s approach to safety or security. If anything, the move to a for-profit Public Benefit Corporation should make you a bit less concerned about OpenAI’s governance: non-profit governance is a far less sophisticated and robust area of the law than traditional corporate governance.

It is true that Murati’s departure, in particular, raises eyebrows. She briefly served as interim CEO of OpenAI after Altman was ousted from the company last year. And according to Altman’s own note to OpenAI staff about her departure, she informed him only hours before the rest of the company learned of the news. All of this points to there being some kind of drama.

As for McGrew and Zoph: Senior executives leave companies all the time; talent churns through Silicon Valley firms constantly. Perhaps these senior executives left because Altman squeezed them out for being insufficiently loyal, or not supporting his restructuring plan. But they may just as likely have left for entirely different reasons. There’s the simple fact that working at OpenAI is surely a grind.

Then there’s the possibility that OpenAI now sees achieving ‘AGI’ as an engineering, business strategy, and political problem rather than as a scientific problem. Perhaps they now feel as though the scientific breakthroughs are in place, or at least within sight of being accomplished, and now it is “just” a matter of building the data centers, doing immense amounts of engineering work, striking deals, and conducting delicate geopolitical footwork. One can understand how this would require a different kind of organization, and a different leadership team, to be successful. One can easily imagine how the senior executives who left might not want to do that work.

My point is not that the alternative explanation I’ve offered is correct—instead, it’s that there are many explanations for this particular fact pattern, and it’s wiser to judge companies by their actions rather than what people say on social media.

Additional thoughts on o1

My piece last week about OpenAI’s o1 series of models argued that the models challenge our assumptions about what a “frontier” AI model is, and thus challenge laws that attempt to regulate “frontier” models. But there are some additional thoughts I didn’t share:

• Jaime Sevilla from Epoch AI made a compelling argument on X that I am wrong about this. He argued, in essence, that using additional “thinking time” on a smaller model can simulate the capabilities of a next-generation larger model. This would give us more concrete understanding as to what next-generation models might be capable of in advance. Ultimately this is an empirical question, so we do not currently know the answer. But I will be very interested to see how o1 (which seems to basically be GPT-4o with OpenAI’s fancy new posttraining methods) compares to the “base” version of a next-generation model like GPT-5, Claude 4, or Llama 4.

• For those concerned about the Nvidia’s dominant position in the AI chip market, the o1 series may create new opportunities for competition. There are American startups working on specialized chips designed exclusively for the purpose of running language models faster than even Nvidia’s chips can. Some of these startups claim their products run language models up to 20 times faster than Nvidia chips. Remember that a chip that can run models like o1 faster is essentially giving the model more thinking time; every time we make chips that can run language models more quickly, we make a model like o1 “think” that much faster. It is also worth noting that Microsoft/OpenAI have worked on dedicated chips for language model inference.

• Another issue I did not write about with the o1 models is OpenAI’s decision to keep the models “chains of thought” hidden from users. When you ask an o1 model a question, it will “think” for a while (it seems to range from a few seconds to maybe a maximum of 2-3 minutes), and then give you its answer. As the user, you can see a summary of this thinking (itself written by another AI model), but you do not get to see the model’s actual thinking.

  I suspect that OpenAI’s primary motivation for this is commercial: those chains of thought are valuable training data! Already, OpenAI and other closed-source model providers are dealing with the problem of “distillation,” when is when another developer uses your model’s outputs to train their own, smaller model. Distillation is not itself illegitimate—models like GPT-4o mini and Anthropic’s Claude 3.0 Haiku are likely distillations of larger models made by those companies. But when another developer distills someone else’s model, that is, at best, a kind of grey area. It is expressly against the terms of service of companies like OpenAI and Anthropic.

  OpenAI also argued there is a safety angle to hiding the chains of thought. Basically, in order to supervise the models’ thinking, OpenAI applies minimal alignment training to the chains of thought themselves. That way, if a model is trying to deceive its users, it will “say so” in its chain of thought. OpenAI wants to be able to supervise the models’ thinking without exposing an unaligned thought process to customers.

  I am skeptical of this, to be frank. First, if models really are deceiving users or having otherwise “unaligned” (which could just mean politically incorrect) thought, it is the users’ right to know. If a model is trying to manipulate a user, shouldn’t the user have a right to know that fact? Isn’t the damage done by a user knowing mostly to OpenAI’s brand, rather than to the safety of the user? Isn’t it more dangerous if a model is attempting to manipulate a user and OpenAI hides this as a matter of corporate policy? And furthermore, if more advanced versions of these models are one day being used to do high-stakes tasks, like designing novel medicines, won’t the corporations using it for those purposes demand to know exactly what they model thought at every step of the way?

  Right now, the risk of user manipulation is not that high (though there is some evidence that o1 manipulated users in pre-deployment testing documented in OpenAI’s technical report), and these models cannot design novel medicines. But I am not sure how tenable OpenAI’s approach is in the medium to long term.