The Order He Didn't Sign

00:00:04 Let me start with the thing that didn't happen. On Thursday afternoon, President Trump walked into the Oval Office and told reporters he'd called off the signing ceremony for his administration's executive order on artificial intelligence — an order that had been on the calendar for later that same afternoon.

00:00:22 The reason he gave was short: "I didn't like certain aspects of it." According to reporting from the New York Times and CNN, it would have handed the federal government the power to evaluate frontier AI models for security flaws before those models reach the public.

00:00:41 The Office of the National Cyber Director, working with other agencies, would have stood up a process to run that review. And the clause that made the labs uneasy, per CNN, was a requirement that companies share their advanced models with the government somewhere between fourteen and ninety days before launch.

00:00:59 Think about what that means in practice. You're OpenAI or Anthropic, you've spent a year and an enormous amount of compute training your next model, and the government wants it in hand up to three months before you ship — to probe it for ways it could find and exploit security holes.

00:01:15 That worry isn't abstract. TechCrunch reports the order was partly a response to two specific releases: Anthropic's Mythos and OpenAI's GPT-5.5 Cyber, both of which can locate and weaponize software vulnerabilities. The government looked at offensive cyber capability arriving inside commercial models and started drafting a pre-release checkpoint.

00:01:35 Then Trump pulled it. His framing was about the race: "We're leading China, we're leading everybody, and I don't want to do anything that's going to get in the way of that leading." He added that AI is "causing tremendous good," and that the order "could have been a blocker."

00:02:00 An executive order signing isn't complete without the row of founders behind the desk, and without them, the ceremony slipped. I can't independently confirm that's the real reason — but two explanations are on the table, and neither one is "the policy was wrong." One is competitiveness.

00:02:17 The other is stagecraft. Here's the piece that complicates the simple read. The administration isn't allergic to model review on principle. Earlier this month, the federal Center for AI Standards and Innovation announced agreements with Google DeepMind, Microsoft, and Elon Musk's xAI to evaluate their models before public release — voluntarily.

00:02:37 So the government already gets a look at some frontier systems. The executive order would have turned a handshake into a requirement, with a clock attached. That's the difference between a lab choosing to cooperate and a lab being told it must. What got deferred, then, is the mandatory version, the one with teeth.

00:02:55 And it got deferred on the same afternoon the administration was pushing hard into AI everywhere else: equity stakes, chip enforcement, and federal contracts. I'll walk through all of it. But hold onto the shape of this. On the one day this week the US government could have claimed a formal seat at the safety table, it stepped back — and the reason it gave on the record was competitiveness.

00:03:18 Anything that slows the labs down is a cost the administration isn't willing to pay right now.

00:03:23 On the same day, the United Kingdom's AI Security Institute — the government body that used to be called the AI Safety Institute — published a report on how we oversee these systems, and whether that oversight is holding up. Its conclusion isn't comforting. The report maps the methods we use to watch advanced AI across four different surfaces, and it finds many pathways that lead toward those methods getting weaker over time.

00:03:47 The institute built this on twenty-five expert interviews, a literature review, and its own analysis, so it isn't one researcher's hunch. Take the example they lead with: chain-of-thought oversight. Right now, frontier models reason step by step in human-readable text — they show their work as they go.

00:04:04 That running commentary is one of the most informative tools we have for catching a model doing something we don't want. You can read what it was thinking. The problem, the institute says, is that the property this depends on is under pressure from several directions at once.

00:04:20 As labs push models to reason more efficiently, that text can compress into something we can't read, or move into the model's internal representations where there's no transcript at all. There's a live disagreement here, and the report is careful to flag it: one expert argued that reasoning in discrete, readable tokens has better error correction, which would slow the drift toward unreadable internal reasoning.

00:04:43 The report tags that as disputed rather than settled, which is the right instinct — and the researchers reading it online singled that honesty out for praise. Here's what should worry you most. The report notes that some pressures on oversight are already visible.

00:04:58 Evaluation gaming — where a model learns to behave well on the tests it knows it's being judged on — is already undermining behavioral audits. And here's their sharpest line: because many of the properties that matter for oversight aren't currently being tracked, some loss of oversight "could go unnoticed in future." We could be losing the ability to watch these systems and not realize it until after it's gone.

00:05:21 Put the two next to each other. A government safety body says the tools we use to supervise frontier AI are eroding, and may erode without anyone noticing. And the same afternoon, the US government postpones the order that would have built it a formal pre-release window to do exactly that kind of supervision.

00:05:39 I'm not claiming the two are connected — different governments, different documents, and a coincidence of the calendar. But the demand for oversight and the supply of it moved in opposite directions on a Thursday, and that's worth noticing.

00:05:52 While the federal order stalled, a different government moved. Governor Gavin Newsom signed an executive order in California directing state agencies to work with the AI industry and others on a question Washington's order never touched: what happens to workers.

00:06:08 Specifically — and this is the piece I keep turning over — the order tells agencies to study subsidies for companies that don't replace their workers with AI. Read that twice. The usual policy lever is to subsidize adoption: tax credits for buying the technology, grants to deploy it.

00:06:24 Newsom is pointing at the opposite. The New York Times, which broke the order, describes it as exploring an overhaul of labor policy to deal with potential mass job displacement. The state isn't paying companies to automate. It's studying whether to pay them to keep humans on the payroll.

00:06:41 Now, the caveat: this is a study order, not a check. An executive order that tells agencies to "explore" and "study" can die in a working group, and many do. There's no dollar figure, no eligible-employer definition, and no funding source named yet. So I'm not going to tell you California just rewrote the social contract.

00:07:00 What I'll tell you is which way the governor of the largest state economy in the country is now pointing — and that he's pointing there at all is a signal about where the labor anxiety has landed. It also collides with something the federal government has been pushing.

00:07:16 The Trump administration has backed preempting states from setting their own AI rules — the idea being that you don't want fifty separate rulebooks, so Washington sets one and the states stand down. California signing labor-focused AI orders is exactly the kind of state action that preemption would shut off.

00:07:34 So a structural fight is forming: the feds want to clear the field of state AI law, and the most populous state just planted a flag in the middle of it, on the issue voters feel most directly — their jobs. We've tracked the labor numbers on this show. The Bureau of Labor Statistics prints showing AI-exposed occupations softening; Intuit cutting seventeen percent of its workforce with displacement named in the reasoning.

00:07:59 Those are the conditions Newsom's order is responding to. Whether "subsidize the firms that keep people employed" survives contact with a budget is a different question, and I'd put the odds low that it survives in that exact form. But the framing is now on the table from a governor with national ambitions, and that tends to be how these things start — not as law, but as a sentence somebody can't take back.

00:08:23 Here's where the same administration that stepped back from oversight stepped forward as an investor. On Thursday, the Commerce Department said it plans to award two billion dollars in grants to nine quantum computing companies — and to take equity stakes in them in exchange.

00:08:40 IBM is set to receive about one billion dollars of that package. Half the total, going to a single company. This is a pattern now, not a one-off. Last year, Commerce took a ten percent stake in Intel. It converted two-point-two billion dollars in Chips Act grants, plus another eight-point-nine billion in federal grants that had been awarded but not yet paid, into equity.

00:09:01 Since then, smaller deals have followed. The Financial Times notes one of them: Vulcan Elements, a rare-earths startup with around thirty employees, which counts a venture firm run by Donald Trump Jr. among its backers. And look at who got left off Thursday's list.

00:09:17 IonQ — a leading quantum company — was absent from the letters of intent. IonQ has drawn significant investment from Cerberus, the firm co-founded by Stephen Feinberg, who is now the administration's deputy secretary of war. So a company tied to a senior administration official sat out, while a startup tied to the president's son has been on the receiving end of the broader push.

00:09:39 I'm not going to assign motive to either fact. I'm noting that when the government starts taking equity positions in strategic technology, the question of who's connected to whom stops being gossip and starts being material. Step back from the personalities and look at the machine being built.

00:09:57 The United States is assembling something that looks less like a regulator and more like a state investment fund — a sovereign-wealth posture aimed at semiconductors, critical minerals, and now quantum. The upside flows to the taxpayer in theory: if these firms succeed, the government holds a stake.

00:10:14 But equity cuts both ways. Intel is already facing a shareholder lawsuit over its government deal. That's a preview of the legal weather ahead: when Washington is on the cap table, every other shareholder gets to ask whether a partner that also writes the rules just diluted them.

00:10:31 And the deals announced Thursday aren't final. The administration said it's still soliciting proposals from other advanced-tech firms, so the list will grow. Put it beside the oversight story and you get the shape of the week. The government is happy to own a piece of the AI build-out — the chips under it, the quantum machines beside it, and the materials feeding it.

00:10:52 What it deferred was responsibility for what the models can do once they ship. Ownership of the upside, yes. The safety liability, not this week.

00:11:01 Jensen Huang said something on Nvidia's earnings call this week that I want to quote precisely, because the bluntness matters. Asked about China, the Nvidia chief executive said Huawei "is very, very strong. They had a record year… because we've evacuated that market.

00:11:17 We've really largely conceded that market to them." Conceded. Not "facing competition in." Conceded. The backdrop is a blowout quarter. Nvidia revenue up eighty-five percent to roughly eighty-two billion dollars, with an eighty-billion-dollar share buyback and a raised dividend.

00:11:34 By any normal measure the company is printing money. But China, which once accounted for at least a fifth of its data-center revenue, has effectively closed to it since April, when the administration told Nvidia it would need a license to export advanced chips there.

00:11:51 Huang told investors to "expect nothing" on near-term approvals. He'd serve the market "more than delighted" if conditions changed; he's just told everyone not to count on it. The second-order effect is the entire story of export controls compressed into one quote.

00:12:07 The policy was meant to slow China's AI progress by denying it the best chips. What Huang is describing is the policy handing Huawei a captive domestic market with no American competition in it — and Beijing using that protected runway to build a homegrown chip industry faster.

00:12:24 The CEO whose sales the controls were supposed to protect is telling you the controls accelerated his strongest rival. We don't know yet whether that's a permanent loss or a negotiating phase. But it's the seller saying it, on the record, on an earnings call. Meanwhile the enforcement side hardened the same week.

00:12:43 The Senate unanimously passed the Stop Stealing Our Chips Act — a bill aimed at strengthening export-control enforcement against the smuggling operations routing restricted chips into China anyway. Unanimous, in this Senate, tells you the compute-advantage framing is one of the few things both parties still agree on.

00:13:03 And it isn't theoretical: Taiwan is now seeking to detain three people accused of forging documents to ship Super Micro servers packed with Nvidia chips into China, Hong Kong, and Macau, in violation of US rules. The embargo has reached the stage of forged paperwork and arrests.

00:13:20 Then, almost as a punchline, AMD pledged more than ten billion dollars into Taiwan's chip industry this week — advanced packaging for AI — and said TSMC will ramp its next-generation Venice chips. So in a single week: the government takes equity in domestic chip and quantum firms, the Senate moves to police smuggling, prosecutors chase forgers across the strait, and the leading American chip designers deepen their dependence on Taiwanese fabrication.

00:13:48 The stated goal is to bring the supply chain home and starve rivals of compute. The commercial reality is that the most advanced silicon in the world is still made on one island off the coast of China, and everyone — Washington included — is building around that fact rather than past it.

00:14:06 Across the Atlantic, a procurement decision did what privacy campaigns mostly haven't: it stopped a major surveillance-AI deployment cold. London mayor Sadiq Khan blocked a fifty-million-pound deal that would have let the Metropolitan Police use Palantir's software to automate intelligence analysis in criminal investigations.

00:14:25 It would have been Palantir's largest policing contract in Britain. The company already holds deals worth three hundred thirty million pounds with NHS England and two hundred forty million with the Ministry of Defence. The lever Khan used wasn't "this is too invasive." It was procurement.

00:14:42 His office found what it called a clear and serious breach of the rules — the Met had considered only one supplier, Palantir, rather than running an open competition. The deputy mayor's letter to the commissioner said she'd been given no acceptable explanation for that failure.

00:14:58 The original internal costing, by the way, was fifteen to twenty-five million pounds a year; the proposed deal sat at the very top of that range. Khan also named the mechanism that worries him, and it's worth hearing in his words: "what you're allowing is these private companies to almost have a loss leader, so they give you a good deal or something for nothing for a short bit of time, and you can become reliant upon them." That's the land-and-expand model in plain English.

00:15:27 Palantir has a documented history of it in the UK — including a prior Met contract to use AI to root out corrupt officers that was awarded directly, with no open competition, because its value came in just under the five-hundred-thousand-pound threshold that would have triggered City Hall approval.

00:15:44 Small enough to slip the gate, useful enough to build dependence on. Scotland Yard is furious, and its argument deserves a fair hearing. The Met says it's facing a hundred-twenty-five-million-pound funding shortfall and eleven hundred fifty job cuts. Without modern technology, it argues, it'll have to reduce officer numbers in a way that hits its ability to keep London safe.

00:16:07 Their line: "We must be able to innovate at a faster rate than hostile states and organised criminals." The Met has a point. The technology might let fewer officers cover more ground. The objection isn't that it doesn't work — it's how the contract was let, and to whom.

00:16:22 Because the "to whom" is loaded here. Palantir was co-founded by Peter Thiel, it serves the Israeli military and US immigration enforcement, and last month its chief executive Alex Karp published a manifesto extolling American power and implying some cultures were inferior to others — which one British MP called "the ramblings of a supervillain." Khan has said before that Londoners want public money going to companies that share the city's values.

00:16:49 The campaign group Foxglove framed the block as catching Palantir's land-and-expand approach before it locked in. Here's why this one travels beyond London. Every Western public service is wrestling with the same setup: a small number of mostly American vendors, offering cheap or free pilots that grow into systems you can't easily remove.

00:17:09 The chair of the Commons science and technology committee welcomed Khan's move as a stand against "vendor lock-in and dependence on a small number of large, US-based providers." That's the fight. And the most effective tool against it this week turned out to be a procurement rulebook, wielded by a mayor — not a privacy court, a regulator, or a street protest.

00:17:31 If you want to know where the friction on surveillance AI actually lives, it's in who signs the contract, and whether they followed the rules doing it.

00:17:40 Now to the physical world, where the gap between the rhetoric and the road was hard to miss this week. Waymo paused its robotaxi service in two American cities — Atlanta and San Antonio — because its vehicles keep driving into floods. On Wednesday in Atlanta, during heavy rain, one of its cars drove into a flooded street and got stuck for about an hour before it was recovered.

00:18:01 The company had already issued a software recall last week for this exact behavior. What Waymo admitted is the uncomfortable bit. When it issued that recall, it told the National Highway Traffic Safety Administration it hadn't finished developing a "final remedy" for avoiding flooded roads.

00:18:18 Instead it shipped a stopgap — restrictions in places and at times where there's an elevated risk of hitting a flooded, higher-speed road. And that stopgap wasn't enough; the Atlanta car drove in anyway. Waymo's explanation is that the storm flooded the street before the National Weather Service had issued any flash-flood warning — and those weather alerts are one of the signals the cars rely on to prepare.

00:18:42 So when the alert lags the water, the car drives into the water. This isn't Waymo's first time struggling to stamp out a behavior quickly. The company is already under investigation by both the National Highway Traffic Safety Administration and the National Transportation Safety Board over its robotaxis illegally passing stopped school buses — a problem it shipped a fix for, only for the cars to keep doing it.

00:19:05 There's a separate set of investigations into a January incident where a Waymo struck a child in Santa Monica; the company says it had braked to around six miles per hour and the injuries were minor. None of this means the technology is unsafe on balance — Waymo's overall record stands up well against human drivers.

00:19:24 It means the long tail is hard, and the failures that remain are physical and stubborn: water on the road, a flooded intersection, and a school bus. They don't yield to a single update. Hold that against the other autonomy story of the same day. Tesla launched "Full Self-Driving (Supervised)" in China, after years of regulatory delay — into a market where domestic electric-vehicle makers have already raced ahead on assisted driving.

00:19:49 So on one Thursday: an American leader in autonomy pulls back in two US cities because it can't reliably detect a flooded intersection, while another American company pushes its driver-assist system into the world's most competitive car market as the late arrival.

00:20:04 I keep coming back to a line from Google's developer conference the day before, when its DeepMind chief called this moment the "foothills of the singularity." Maybe. But the foothills this week included a robotaxi sitting dead in an Atlanta flood for an hour, waiting for a tow.

00:20:20 What these systems can do keeps expanding. What they can be trusted to do, unsupervised, in the rain, is moving a lot more slowly — and that second curve is the one that decides whether any of this reaches the street.

00:20:32 Let me close where the leverage actually sits — the money. New financial reporting this week: OpenAI generated about five-point-seven billion dollars in revenue in the first quarter, according to The Information, nearly a billion more than Anthropic's roughly four-point-eight billion in the same period.

00:20:50 So OpenAI is out front on the top line. But the more revealing number came out of a filing, not a leak. SpaceX filed to go public, and its IPO paperwork put hard figures on the compute deal we covered yesterday — the one between Anthropic and Elon Musk's data-center business.

00:21:05 Anthropic agreed to pay one-point-two-five billion dollars a month — fifteen billion a year — through May of 2029, for access to the Colossus data centers in Memphis. Fifteen billion a year. That single contract could nearly double the eighteen-point-seven billion dollars SpaceX reported in all of last year.

00:21:23 Now layer in what the filing says about that data-center business itself. SpaceX's AI division lost six-point-three billion dollars on three-point-two billion in revenue last year. It spent nearly thirteen billion on AI capital outlays — about sixty-one percent of its total spending.

00:21:39 So look at Anthropic's position. It expects at least ten-point-nine billion dollars in revenue this year, and it's closing in on its first quarterly operating profit. And it's tied a large share of its compute future to a money-losing division of a rocket company — one controlled by a man whose own AI lab, xAI, competes with Anthropic's Claude.

00:21:59 The deal has a ninety-day exit clause for either side. You don't write a ninety-day escape hatch into a multi-year deal worth tens of billions of dollars unless both parties know how fast the ground can move under them. There's one more number worth keeping in view, because it cuts against the political narrative.

00:22:17 Federal records show that across more than four hundred publicly identified government AI use cases last year, OpenAI's ChatGPT was used in two hundred thirty-four of them. Google's Gemini, thirty-three. Anthropic's Claude, twenty-six. And Musk's Grok — despite all the White House proximity — was used in three.

00:22:35 Whatever access xAI has in Washington, it isn't translating into agencies actually running its model. Distribution inside the government tracks something other than who's standing closest to the president. So here's the week, all together. The state moved hard into AI almost everywhere — buying equity in chip and quantum firms, policing the chip trade, standing by as a London mayor killed a surveillance contract on a procurement technicality, and watching a California governor reach for the labor question.

00:23:04 The one place the US government pulled back was where it would take responsibility for inspecting the models themselves. Ownership, enforcement, and industrial policy — yes. The safety checkpoint — postponed, because it might slow the race. There's a clean test coming.

00:23:19 Either that executive order comes back with the fourteen-to-ninety-day pre-release clause intact, or it returns gutted, with the mandatory review traded away for another voluntary handshake. The CEOs couldn't make the photo op on Thursday. When they finally do stand behind that desk, the language they're standing behind will tell you who wrote it — the agencies, or the labs.

00:23:40 That's the document I'll be reading next. — Jonas