Double-blind evals, a $399 robot, and Schwab's altcoin pivot

00:00:04 Google DeepMind this morning announced what they're calling an industry first: a double-blind evaluation of a proprietary frontier model. AVERI, OpenMined, and MLCommons are the other partners. The basic setup is straightforward. Neither Google nor the external evaluators see each other's side.

00:00:24 Google doesn't get to peek at the test prompts ahead of time, so they can't optimize for a known benchmark. The evaluators don't see the model weights or architecture, so they can't reverse-engineer what to probe for. The test ran on Gemini 2.5 Flash-Lite through PySyft, rebuilt from scratch by Koen van der Veen's team over roughly ten months for exactly this kind of secure computation.

00:00:51 The compute overhead is already under five percent, which means the technique is practical enough to scale beyond one-off pilots. The real watch here isn't the headline — it's whether other labs pick up PySyft and run their own double-blind evaluations on their own frontier models.

00:01:10 Benchmark contamination has been a quiet rot in AI for years because public evals get absorbed into training data, and model cards cite scores that reflect familiarity rather than capability. This protocol stops that by design. The open question everyone's asking is about trust in the enclave itself.

00:01:31 If evaluators can't see the prompts and Google can't see the weights, who verifies the secure environment did load the right model? Koen van der Veen said it was the first of its kind to the best of his knowledge. Miles Brundage noted this required both technical and institutional innovation — four organizations coordinating across what are usually competitive boundaries.

00:01:57 Whether this becomes standard practice depends on whether Google publishes results that look bad as well as results that look good.

00:02:06 Hugging Face had a big day. They unveiled Microduck from their Pollen Robotics team — a $399 open-source robot that can walk, pick things up, get back up when it falls, and roller-skate. It's taught through reinforcement learning, not hardcoded behaviors. Clem Delangue made the announcement to about seventeen hundred views and seven thousand likes, which is substantial for a hardware launch in this space.

00:02:34 It packs a camera, a microphone, and LiDAR into a tiny chassis. The conversation around it split exactly where you'd expect. Half the responses were excitement about affordable robotics for research and education. The other half, which turns out to be the more important thread, pointed out what the device contains and where it lives.

00:02:58 Hexablob put it cleanly: open-source is the only reason a camera, microphone, and LiDAR in someone's living room is acceptable right now. You can audit what the robot sends home. A closed consumer robot with those same sensors would face scrutiny that this doesn't have to deal with.

00:03:17 People are going to start asking about data flows for these things eventually. Whether Microduck matters goes down to one thing: where the open part stops. Are CAD files, firmware, and training code all released? Or is it policy checkpoints plus an SDK? At three hundred ninety-nine dollars, that difference determines whether people can fork the hardware and modify it.

00:03:43 There's also the METR report on Hugging Face that came up in conversation today, which reminded me of something Ethan Mollick flagged: people are ascribing way too many human motivations to agents and organizations based on chain-of-thought studies run by overwhelmed teams.

00:04:02 It's useful to separate what these launches do from the narratives people build around them. But Thomas Wolf ordering seven hundred agents to go attack Hugging Face — that line from today has a certain energy to it, even if it reads as humor.

00:04:19 Anthropic's Claude team announced Chat SDK alongside managed agent infrastructure. The SDK drops a single type-safe direct-message handler into your setup, backing it with adapters for Slack, Teams, Discord, web interfaces, and a few others. Managed Agents runs the harness, session management, and memory server-side — each conversation is one persistent session.

00:04:45 The part I want to flag is the Vercel integration. They released a cookbook connecting Claude Managed Agent to Vercel's Chat SDK, giving agents access to a universal chat layer you can drop into existing platforms. You can also bring your own Vercel Sandbox for tool execution or use the built-in environment.

00:05:07 The self-hosted sandbox option is what makes this useful for organizations with data residency requirements. Most managed agent announcements promise flexibility and stop there. This one ships actual code you can run yourself. Aravind Srinivas shared a tweet today titled just The Agentic Cloud, though he didn't add body text to it — which is its own kind of signal.

00:05:32 The phrase has been circulating as shorthand for the idea that agent infrastructure is becoming the next platform layer, and seeing it come from someone who builds these systems feels more like a naming convention than a rumor.

00:05:49 Charles Schwab is adding Solana, Avalanche, and Chainlink trading to their crypto accounts. Watcher.Guru broke the story this morning, and the reaction was exactly what you'd expect from the intersection of finance Twitter and crypto Twitter — a mix of validation and skepticism.

00:06:08 Schwab manages roughly thirteen trillion dollars in assets. When a brokerage at that scale adds support for three major altcoins beyond Bitcoin and Ethereum, it's not a niche signal anymore. This is wealth management infrastructure reaching into the broader digital asset layer.

00:06:27 The practical question here is which Schwab clients will get access and what the fees look like. The announcement doesn't specify rollout timelines or minimum balances. For now, it's a signal about where institutional crypto exposure is heading rather than an immediate change in behavior for most people.

00:06:49 Four stories today that all sit at the boundary between building and governance. The double-blind evals are infrastructure disguised as methodology — they only work because someone rebuilt PySyft. Microduck raises questions about consumer hardware with sensors that have been a privacy controversy for years, answered by open-source code you can audit.

00:07:11 Claude's Chat SDK is backend glue — adapter counts, persistent sessions, self-hosting options — that decides whether agent platforms get adopted or abandoned. And Schwab adding altcoins is institutional behavior shifting at the margin. The two details that will decide how these stories play out over the next few weeks are whether the double-blind results get published regardless of what they show, and whether Microduck's open source extends to firmware and training code past the initial policy checkpoints.

00:07:44 Those choices will set the boundary for both.