The Co-Mathematician, The Economist, And The Watermark

00:00:04 It's Friday, May 8th. I'm Jonas Vale, and this is IMPULSE. Three stories sit on the table tonight, and they don't share a press release, but they do share an argument. Google DeepMind announced a multi-agent system called the AI co-mathematician, which scored forty-eight percent on the hardest tier of FrontierMath in fully autonomous mode.

00:00:23 The same week, DeepMind quietly stood up something it's calling an AGI Economics team, and named the labor economist Alex Imas as its director. And this morning, the European Commission opened a public consultation on the watermarking and disclosure rules of the AI Act, with the clock running to an August second compliance deadline.

00:00:43 Three institutional moves, three different rooms, one underlying question. Who gets to price what AI does next — the mathematicians, the labor economists, the regulators, or the labs themselves? On a normal Friday, you'd cover one of these and let the other two ride.

00:00:59 Today they're sitting on the same shelf. So we're going to take them in turn, and then look at the medical agents, the alignment paper that's making the rounds, and a piece of audit work on regional model bias that will quietly matter more than its citation count suggests.

00:01:15 Let's start with the math.

00:01:16 Pushmeet Kohli, who runs research at Google DeepMind, posted the announcement at about eleven in the morning Pacific. Quote: "The future of math is mathematicians and AI agents working together. Very pleased to introduce Google DeepMind's AI co-mathematician — a multi-agent system designed to actively collaborate with human experts on open-ended research mathematics." The mathematicians who tested it, he said, worked across group theory, Hamiltonian systems, and algebraic combinatorics, which is a pretty wide spread.

00:01:47 And then the number that made the rest of the post matter. In autonomous-mode evaluation on the FrontierMath Tier 4 problems, the system scored forty-eight percent. Kohli called it a new high score among all AI systems evaluated. A word about that benchmark, because it deserves it.

00:02:04 FrontierMath was built by Epoch AI in collaboration with around sixty professional mathematicians, and Tier 4 is the hardest of four difficulty bands. The problems are described by their authors as the kind of thing that would take a domain expert hours to days to solve, with original ideas required, not look-up.

00:02:22 The point of the benchmark, when it was released, was to be unreachable by current models. Last year, the best published number on Tier 4 was in the low single digits. The number Kohli claims today is forty-eight. Now, the qualifiers. We don't yet have a paper, a system card, or an independent evaluation.

00:02:40 We have a screenshot of a benchmark chart, the announcement post, and the fact that mathematicians who held the agent in their hands described the experience favorably. I'd like to see the writeup, an Epoch AI confirmation of the methodology, and the agent run against a held-out set Kohli's team didn't get to see in advance.

00:03:00 None of those have landed yet. Until they do, read this as a strong claim from a credible lab, not a settled fact. The framing word matters. Co-mathematician — not solver, autonomous mathematician, or replacement. The hero workflow DeepMind is selling is a human researcher and a multi-agent system collaborating on open problems, with the agent able to handle large stretches independently when asked.

00:03:24 That framing reflects something a few mathematicians have been saying for the last six months, including Terence Tao, who has been blogging about using these tools to chase intuitions and check small lemmas. He's been clear that the agent doesn't replace the mathematician, but it changes which questions are cheap to ask.

00:03:43 A forty-eight on Tier 4, if it holds up, changes which questions are cheap to ask by an order of magnitude. What does this mean for the wider world? Probably less than the headline suggests, and more than the cynics will allow. Math papers are not a labor market in the conventional sense, and pure mathematicians are not, despite their occasional protests, a guild that lobbies Brussels.

00:04:06 But mathematics sits upstream of physics, of cryptography, of optimization, of every quantitative discipline that gets turned into infrastructure ten or twenty years downstream. If a co-mathematician system can chip away at open Tier 4 problems, the next decade of cryptographic standards, of materials science, and of theoretical machine learning gets a different default collaborator.

00:04:29 It happens quietly at first, and then less quietly. That's the story to track.

00:04:33 On the same day Kohli posted his benchmark, Alex Imas posted a different kind of announcement. Quote: "Some news. This week I am starting at Google DeepMind as Director of AGI Economics on Shane Legg's team." His charter, in his own words: "My team will study how frontier AI could reshape the economy — what happens to work and labor, how wealth and power are distributed, how institutions adapt, how AI agents shape markets, and what kinds of models can help us reason clearly about futures that may look very different from the past."

00:05:07 Shane Legg is one of DeepMind's three co-founders, and the only one of the three whose public output for years has been about AGI as a research target rather than as a product. The team Imas is joining sits on Legg's side of the org, not on the policy side, not on the comms side, and not under the labs' chief economist if they have one.

00:05:27 So this isn't a public-affairs hire. It's a research hire, with a labor economist's CV running a team that's going to publish. Imas himself comes out of Chicago Booth and a behavioral-economics lineage. He has spent a long time on questions of how people make decisions under uncertainty, how labor markets process information, and how institutions absorb or reject technological change.

00:05:50 The fact that Google DeepMind has decided that's the discipline they need next, sitting next to ML research and not on a separate floor, says something about how they think AGI will be understood inside the lab. Not as an alignment problem, not just as a capabilities curve, but as an economic phenomenon that needs to be modeled with the tools of economics.

00:06:11 A cynical read is available, and I won't pretend otherwise. A frontier lab funding its own labor economists puts those economists in a position to publish the early studies that policymakers and journalists will reach for in 2027 and 2028 as the first wave of displacement data hits.

00:06:28 That's a soft form of agenda-setting that the labs have, until now, been content to outsource to Stanford, MIT, and the various AI policy think tanks. By bringing it in house, DeepMind is taking the same step OpenAI and Anthropic have taken on safety research over the last three years.

00:06:45 Whose papers count as the canonical reference is itself a kind of power, and the labs have decided they want a seat in the room when those papers get written. The charitable read is also available. AGI economics, as a field, doesn't really exist yet. Nobody has good models for what happens when the marginal worker is a software process you can replicate, charged not at a wage but at a compute-rental rate.

00:07:09 The economists who've tried — Daron Acemoglu, David Autor, Anton Korinek among them — have done it without access to the deployment data the labs collect. Putting a labor economist inside the lab gives at least one team access to that data, with the obvious caveats about what they'll be allowed to publish.

00:07:27 I'd want to read this team's first three papers — not the announcements or the blog posts, the papers themselves. They will tell us whether DeepMind hired an economist to steer the conversation or to actually figure out the answer.

00:07:41 While Imas was packing for his first day, Ethan Mollick, who teaches at Wharton and writes more clearly about AI's labor effects than almost anyone, posted what reads like a thesis statement. Quote: "A machine that can replace all US white collar work by 2035 will, in no way, be allowed to replace all US white collar work by 2035." In the parent thread, he laid out the political-economy argument.

00:08:07 Professions with guilds — the Bar, the American Medical Association — will preserve human-required activities through legislation. Professions without — consultants, coders, analysts — will not enjoy the same protection. And the loudest pushback, when it comes, will come from highly connected, organized, wealthy white-collar workers who decide their jobs are at stake.

00:08:30 A reply in that thread, from a marketing executive named Anees Merchant, sharpened the point. Quote: "The political economy answer always beats the technical capability answer. Same thing happened with port automation in the US, with telemedicine across state lines, and with self-driving trucks.

00:08:49 Capability arrives early, the legal and labor reaction sets the actual deployment." That's the right historical frame. AI is not the first technology to outpace its institutional clearance, and political economy is not a sideshow to deployment. Political economy is the deployment.

00:09:07 Here's why I'm pairing Mollick with Imas, and with a paper that landed on arXiv this morning under the title, "Who Prices Cognitive Labor in the Age of Agents? A Position on Compute-Anchored Wages." The author, Siqi Zhu, makes an argument I haven't seen put this cleanly before.

00:09:25 Quote: "Agents are not labor. They are a production technology that converts compute capital into effective units of cognitive labor. Once this is recognized, the elastic-supply margin that anchors the equilibrium wage migrates from the labor market to the compute capital market."

00:09:47 In their model, the competitive human wage is bounded above by relative human-to-agent productivity, multiplied by the compute intensity of one effective agent-labor unit, multiplied by the rental rate of compute capital. Translated into plain English: if you and an agent are substitutes for the same task, your wage is capped above by how much it costs to rent the compute the agent needs.

00:10:12 The price-setter for cognitive labor, the paper argues, is no longer the labor market. It's the GPU rental market. A caveat. This is a position paper. It's a theoretical argument with a textbook-style derivation, not an empirical claim, and it does the thing economists like to do where everything follows from elegant premises that may or may not match the world.

00:10:36 The premises that have to hold are that agents and humans are real substitutes on the relevant tasks, that the compute market is competitive, and that the legal and political shields Mollick is pointing at don't intervene. Those are three big assumptions, and the lawyers, the unions, and the AMA are about to spend the next decade making sure at least one of them fails.

00:11:00 But as a frame for what Imas's team will probably end up modeling, and what regulators will eventually be staring at, the Compute-Anchored Wage idea is hard to unsee once you've seen it. If even part of the cognitive economy clears at compute-rental rates, then NVIDIA's pricing schedule, the hyperscalers' depreciation schedules, and the grid capacity in places like West Texas and South Korea start showing up in wage data through a transmission channel labor economists have not had to model before.

00:11:32 That's a story this show is going to keep returning to.

00:11:35 Switch hemispheres. This morning, May 8th, the European Commission's Digital Strategy office opened a public consultation on the draft guidelines for transparency obligations under the AI Act. The window runs from today until June 3rd. The rules in question apply on August 2nd, which gives Brussels under three months to absorb the feedback and ship the final guidelines.

00:11:57 The relevant clause is Article 50. It does four things. One, providers of any AI system that interacts with people have to inform users when they're interacting with an AI system. Two, generative AI systems have to implement machine-readable marks — watermarks, in plain English — so that synthetic content can be detected as AI-generated or manipulated.

00:12:17 Three, deployers have to inform people when they're being shown deepfakes, or AI-generated publications on matters of public interest. And four, biometric categorization and emotion-recognition systems have to disclose themselves to the people they are pointed at.

00:12:33 This is the obligation that touches every consumer-facing AI in the European single market, regardless of whether the underlying model gets classified as high-risk under the rest of the AI Act. The consultation page is unusually direct.

00:12:46 Quote: "Providers of AI systems will have to inform users when they are interacting with an AI system and implement machine-readable marks in generative AI systems to enable the detection of synthetic content as AI generated or manipulated." That sentence, if it survives the consultation, is the legal hook for watermarking the output of every commercially deployed generative model in Europe.

00:13:09 Watermarking is hard. The technical work on it has been mixed. Google's SynthID embeds signals in image and audio generations that survive cropping, compression, and some classes of paraphrase. Meta and OpenAI have shipped variations on the same idea. None of them are perfect, and academic work has shown most existing watermarks can be removed by an attacker willing to spend a bit of compute.

00:13:32 The honest answer about watermarking, today, is that it's a probabilistic deterrent, not a guarantee. The Commission knows this — the consultation explicitly contemplates a parallel voluntary Code of Practice on AI-generated content, which is the EU's polite way of acknowledging that the binding rule may have to soften where the engineering can't deliver.

00:13:53 This lands the day after we talked about the EU's Digital Omnibus, which is deferring the high-risk rules for general-purpose models. The pattern that's emerging is not that Brussels is stepping back. It's that Brussels is stepping back on the parts of the law that fight with Mario Draghi's competitiveness agenda, and stepping forward on the parts that fight with consumer-facing harm.

00:14:16 The high-risk rules and the foundation-model thresholds get deferred. The chatbot disclosure rules, the deepfake labeling, and the watermarking move on schedule. If you ship a generative model into Europe, August 2nd is your deadline. Between now and then, two things to watch.

00:14:32 First, whether the Commission's final guidance lets watermarking happen at the platform layer, or whether it has to be embedded by the model provider. The two answers route the obligation to very different parties, and the lobbying around that question is going to be loud.

00:14:47 Second, whether the deepfake-labeling rule includes news-organization carve-outs, the way the original directive hinted at, or whether the journalism exception narrows during consultation.

00:14:58 Two medical items today, both worth taking together. The first is a paper that landed on arXiv this morning under the title "A Versatile AI Agent for Rare Disease Diagnosis and Risk Gene Prioritization." The system is called Hygieia, after the Greek goddess of health.

00:15:15 The author list runs to twenty names and includes researchers from Yale School of Medicine and Duke-NUS Medical School, with corresponding authors at both institutions. That collaboration matters. Yale and Duke don't lend their names to a multi-agent diagnostic system unless they have held it in the clinic for a while.

00:15:35 What does Hygieia do? It's a router-based, knowledge-enhanced multi-agent system that integrates phenotype features, genetic profiles, and clinical records, and outputs a diagnosis with a confidence score. It's specifically tuned for rare diseases, where pattern matching across phenotype-gene-clinical data is exactly the task humans struggle with most.

00:15:57 The headline number from the paper, in the authors' words: Hygieia delivers "superior diagnostic performance compared to physicians, with an improvement from twelve to sixty percent." That's a wide band, and the wide band tells you something. On easy rare-disease cases, the lift is modest.

00:16:16 On the hard cases — the ones where a patient has been bouncing through specialists for years without a diagnosis — the lift is much larger. A caveat, because we're in medicine. The benchmarks here are constructed sets, validated retrospectively, with clinician collaborators in the loop.

00:16:34 The next step, the one regulators will look for, is prospective trials. Mount Sinai's PhysicianBench, which we covered earlier this week, was a useful counterweight: it found a meaningful gap between AI on standardized vignettes and AI on actual patient encounters.

00:16:50 I'd put Hygieia in the same evidentiary category for now. Promising, validated by named medical schools, awaiting prospective evidence. The second item is from the FDA. Today the agency issued its seventh approval under the National Priority Voucher pilot program, for a drug called Bizengri — generic name zenocutuzumab-zbco — to treat NRG1 fusion-positive cholangiocarcinoma.

00:17:14 That's an ultra-rare, aggressive cancer of the bile ducts. Without getting into the chemistry, this is the kind of drug whose approval matters most to the few hundred patients a year who would otherwise have no treatment, and whose review timelines historically stretched for years.

00:17:32 The priority voucher pilot is FDA's mechanism for compressing those timelines. It's also the regulatory testbed where AI-assisted submissions and AI-augmented evidence packages are starting to plug in. We talked Wednesday about Elsa 4.0, FDA's reviewer-side language model, and the HALO repository it ingests.

00:17:51 Elsa is sitting on the reviewer's desk while priority vouchers compress the calendar on the sponsor's side. The Bizengri approval is the seventh data point in a pipeline that, by next year, is going to look very different than it did in 2023. One thing in particular to track.

00:18:08 The voucher program's first six approvals were broadly seen as fast but not corner-cut. If the seventh through fifteenth start looking like they were rubber-stamped — if the labels start carrying odd uncertainty, or if post-marketing safety signals show up on a batch of these drugs — that's the moment the AI-augmented review pipeline gets its first real institutional test.

00:18:32 Until then, this is regulatory infrastructure quietly doing its job.

00:18:36 Now to a paper that's going to upset some plans. The title is direct: "Automated alignment is harder than you think." The authors include Geoffrey Irving, the chief scientist at the UK AI Safety Institute, along with Jacob Pfau and two co-authors. This is essentially an institutional position from the safety side, dressed as an arXiv paper.

00:18:56 It deserves to be read that way. The paper takes aim at what has become the leading proposal for aligning artificial superintelligence — namely, using AI agents to automate an increasing fraction of alignment research as capabilities improve. Anthropic, OpenAI, and DeepMind have all leaned on this idea, sometimes explicitly, sometimes as the unnamed assumption behind their scalable-oversight roadmaps.

00:19:20 The paper's core claim, quoting the authors: "Even when research agents are not scheming to deliberately sabotage alignment work, this plan could produce compelling but catastrophically misleading safety assessments resulting in the unintentional deployment of misaligned AI."

00:19:45 First, optimisation pressure means that when agents make mistakes, the mistakes are concentrated among exactly the cases human reviewers are least likely to catch — because that's what optimisation does. Second, agent errors do not resemble human errors, so reviewers' instincts are calibrated to the wrong distribution.

00:20:04 Third, AI-generated alignment solutions may involve arguments humans simply can't evaluate. And fourth, because agents share weights, training data, and post-training procedures, their outputs are more correlated than human equivalents, meaning you don't get the independence that makes peer review work in human science.

00:20:23 If those four claims hold, the practical implication is severe. The whole scaffolding the labs have been building — automated red-teaming, synthetic-evidence safety cases, agent-assisted interpretability — is leaning on an assumption that the agents producing the safety case are at least neutral observers.

00:20:41 The paper says they probably are not, even when they are not adversarial. They produce systematically biased outputs that reviewers can't catch, on the exact problems where catching the bias matters most. The paper is not nihilistic. It points at two candidates for fixing this.

00:20:57 The first is generalisation, meaning training agents in ways that make their behavior on hard-to-supervise tasks predictable from their behavior on easy-to-supervise tasks. The second is scalable oversight, meaning building processes that let humans evaluate work even when they cannot directly audit each step.

00:21:15 Both are active research areas. Neither is solved. The authors' position is that the labs deploying frontier models on the assumption these problems are tractable need to do the tractability work first, in public, before the assumption gets baked into shipping rationale.

00:21:31 Why does this matter outside the alignment community? Because the regulator's instinct, especially in the United States, has been to defer to the labs' own evaluation processes. The CAISI agreements we covered Tuesday — the pre-deployment screening protocols with five frontier labs — assume the labs can produce credible safety assessments using mostly their own internal tooling.

00:21:53 If those internal tools are themselves agent-assisted, and if the Irving paper is right, the assessments may be telling the regulators what an optimised agent thinks the regulator wants to hear, and not what's actually true about the model. That's a load-bearing assumption I haven't seen anyone in Washington stress-test publicly.

00:22:12 The UK AI Safety Institute just published a paper saying it should be.

00:22:17 Two shorter stories before I sign off, both worth holding next to today's headlines. The first is a paper called "The Geopolitics of AI Safety: A Causal Analysis of Regional LLM Bias." The author, Alif Al Hasan, audits seven instruction-tuned models from different regions: Llama-3.1 and Gemma-2 from the United States, Mistral from Europe, Falcon from the United Arab Emirates, Qwen and DeepSeek from China, and Airavata from India.

00:22:44 Instead of measuring bias the usual way — by counting toxic outputs against demographic prompts — the paper applies Pearl's do-operator to mathematically isolate the causal effect of injecting a cultural demographic into a prompt. What the audit finds is that standard observational fairness metrics overestimate demographic bias, because the topics and the demographics are correlated in the test sets.

00:23:09 The interventional measurement looks different. Quoting the paper: "Western models exhibit higher causal refusal rates for specific demographic groups, whereas Eastern models demonstrate low overall intervention rates with targeted sensitivities toward regional demographics." Translated: American and European models say no more often, and the noes cluster on certain demographic groups.

00:23:33 Chinese and Indian models say no less often overall, but spike sensitivity around their own regional groups. This matters because it makes refusal behavior a quiet form of geopolitics. As global software adopts whichever frontier or open model is closest to hand, a software product built on Qwen will quietly enforce a different content policy than one built on Llama, even when both are fine-tuned for the same downstream task.

00:24:00 Regulators have not started thinking about model choice as content-policy choice yet. They will. The audit work that papers like this enable is going to feed into that conversation, and probably into the next round of EU AI Act guidance after the August deadline passes.

00:24:17 The second item is a smaller paper, but it lands without a press release and matters anyway. A team at the University of North Carolina at Charlotte and the University of Minnesota built an AI-on-CCTV pipeline to evaluate whether temporary pedestrian refuges and curb extensions in Minneapolis actually slowed cars down.

00:24:37 The numbers are concrete. At unsignalized intersections, mean speeds fell up to eighteen and three-quarters of a percent, eighty-fifth-percentile speeds fell up to sixteen and a half percent, and pass-through traffic dropped by as much as twelve percent. At signalized intersections the numbers were similar.

00:24:56 This is the model use case for AI on CCTV: low-cost, evidence-based evaluation of traffic-calming interventions. It gives city planners exactly the kind of data that lets them justify a policy change to a city council. And the same camera-plus-model stack that produces evidence for traffic calming produces evidence for everything else a camera sees.

00:25:18 License plates. Faces, when the camera resolution permits. Pedestrian flow patterns. Cities will adopt this for the safety case and inherit the surveillance case along with it. The way Minneapolis answers the question of what its CCTV pipeline retains, and for whom, will set a template that other cities copy.

00:25:37 Model city ordinances on this in 2027 will tell us which template wins.

00:25:41 That's where I'll leave it tonight. Three institutional moves on the same Friday — DeepMind's co-mathematician, DeepMind's economist, and Brussels's transparency consultation — and three slower-burning stories underneath them on medicine, on alignment, and on the geopolitics of refusal.

00:25:55 What I'm watching next week. First, whether anyone independent of Google DeepMind verifies the FrontierMath Tier 4 score, ideally with a held-out problem set. Second, whether Alex Imas's team publishes anything before the end of the quarter, and what its first methodology paper assumes.

00:26:09 Third, whether the EU consultation gets significant pushback from the major model providers on where the watermarking obligation lands — at the model layer or the platform layer. Those three questions decide which version of this Friday we're remembering by year-end.

00:26:22 I'm Jonas. Have a decent weekend.