Regulators writing rules, evaluators demanding access, and the cost of building tooling from scratch

00:00:04 Friday morning the CFTC filed new rules for crypto transactions and markets through the Federal Register. The filing dropped without much fuss because the Clarity Act stalled in Congress and no one had a better narrative ready. A comment period is now open. Underneath the quiet rollout, regulatory frameworks are developing around crypto markets independent of legislative action, and that's been true for a while.

00:00:34 But this particular filing arrives at a moment when broader questions about AI evaluation governance are also hitting the floor. The relationship between these threads isn't as neat as an editor might want it to be. One is a financial commodities regulator writing rules.

00:00:53 The other involves researchers demanding structural guarantees around how AI companies evaluate themselves. They happen to occupy overlapping concerns — who gets to judge, under what conditions, and who enforces the judgment — without sharing any obvious architecture.

00:01:13 Watcher.Guru broke the story first, pulling over five thousand likes because the crypto community has a strong instinct for when regulatory friction shifts one way or another. The open comment period means market participants will have their say before final rules take shape.

00:01:33 CFTC filing links through reginfo.gov with identifier 1537870.

00:01:38 Arvind Narayanan published a public letter with over one hundred signatories calling for five minimum requirements to make embedded AI evaluations credible. The AI Evaluator Forum organized it and links are on the evaluator forum site. The letter doesn't just ask for independent auditors.

00:01:58 It specifies conditions: evaluators must not be owned or governed by the companies they evaluate, can't have other significant commercial business with them, and shouldn't accept any form of payment contingent on findings. They need full editorial control. Multiple evaluation organizations should cover different risk areas rather than a single auditor per lab.

00:02:25 NDAs are limited to IP protection, customer privacy, security, and public safety — not broad confidentiality that would let companies hide findings behind contracts. The funding mechanism matters most here. Evaluators need a guarantee of continuity even after drawing unfavorable conclusions about the companies they embed with.

00:02:48 Access must also equal what senior internal staff gets during comparable risk assessments. This shifts from vague goals to hard baselines. The AEF-1 standard mentioned in the letter has early adoption already, and these terms are being designed as enforceable standards.

00:03:08 It matters because embedded evaluations are already being proposed as a compliance mechanism, and without specific constraints they risk becoming theater. I'm curious about one thing: who actually funds these evaluation organizations? The letter mentions funding mechanisms that guarantee continuity, but the structural problem of keeping evaluators financially independent from the companies they evaluate remains unsolved in practice.

00:03:39 Nathan Lambert pointed out something that sits at the edge of a larger debate after the HEIF Heist incident was disclosed. The attack compromised OpenAI through libheif — one obscure image library that then cascaded into Slack, Meta, GitHub Enterprise, and more across Rails, Next.js, and ImageMagick.

00:04:01 The open-weight versus closed-model safety argument has been going on for years, usually with abstract claims about who poses more risk. The HEIF Heist makes this concrete: the attack vector was a deployed chat model's API, not an open weights release. Closed models were easier to get started with, more capable in practice, and shipped with leakier safeguards than the argument assumes.

00:04:28 KissonL captured it in the thread: safeguards trained on a deployed chat model can be walked past by techniques that recover base behavior. Open models make that gap visible rather than creating it. The scary production surface isn't the weights. It's every closed API your agents already treat as a trusted tool path.

00:04:51 Rachel St. Clair added that stripping safeguards from open weights is documented and easier; building net-new capability uplift on a specific attack chain is harder. Mahesh KMB raised a durability question — whether openness itself makes misuse harder once open models match frontier capability, or whether the safety argument rests entirely on a capability lag that won't hold.

00:05:18 Shuai Zeng flagged the disclosure lag: open models at least fail publicly. Closed labs can sit on incidents until journalists extract them, and that delay multiplies risk.

00:05:31 Trail of Bits did something interesting this week during an audit for 0xMiden's zero-knowledge virtual machine. The custom assembly language MASM had no existing developer tooling at all. They used Claude and Codex to build a decompiler and static analysis engine from scratch.

00:05:51 The AI-built tools flagged a signature-forgery bug that could have cost millions. One commenter noted: building the tooling from scratch is the unglamorous part nobody budgets for on a custom VM audit. catman nailed what the leverage actually comes from — translating an unfamiliar language into analyzable form, letting analysis expose bugs that manual inspection would struggle to reach.

00:06:20 This doesn't require AI to be perfect here. It just needs to be good enough to bootstrap a human review process. The link between this and HEIF runs deep: Trail of Bits used AI tooling to find bugs in infrastructure humans built but nobody thought to audit. The HEIF Heist found how an AI model could be exploited because its safeguards were leaky.

00:06:46 Both are about tooling gaps — one in analysis and one in defense. The security audit industry is shifting here too. OpenZeppelin was acquired by S&P Global yesterday according to a separate thread, signaling institutional consolidation moving in parallel with this practical shift toward AI-augmented analysis.

00:07:09 The last item today sits in a different register but connects through the same question — when code generation is cheap, what does quality become? Liam Powell wrote about Bend 2 and what he calls the vibe-coding trap. The language asks developers to write laws — specifications that an AI implements and proves.

00:07:30 The demo ships with fifty-eight lines of LAW spec for a simple game. It then requires another four hundred and forty-two lines of proof to verify it. Powell recreated the same verification in SPARK, an established formal verification language, with far less code by using existing formal methods rather than building proofs from first principles.

00:07:54 He told a large language model to recreate the demo in SPARK without guidance and got the result directly. His point isn't about Bend specifically; it's that vibe coding lets you ship a substantial solution before you learn enough to spot the obvious alternative.

00:08:13 He ended up building a whole new language for formal verification without realizing established methods already existed. The real risk here is subtler than speed gains or token costs. You can generate code instantly, so the bottleneck shifts from whether you can build something to whether you should be building it at all.

00:08:35 A large language model will happily implement any architecture you describe and never stop to suggest that what you're building mostly already exists as work you can build on. Powell's article is available at liampwll.com.

00:08:50 The regulatory filings and the evaluation governance letter both point at the same underlying question — who builds the infrastructure for judgment, and under what conditions. The security tooling shift and the vibe-coding critique extend that into how we actually evaluate work once it exists.

00:09:06 Leave the tooling access question on the table for now. Seln.