The Copper Wall and the Open Inference Bet

00:00:04 Most headlines focus on model releases and pricing pages, but the shift that actually changes how you build systems is happening under the floor. Nvidia announced NVLink Fusion partners expanding to include photonics companies like Ayar Labs, Marvell Technologies, and Lightmatter.

00:00:23 At the same time, Meta confirmed via internal memo that it will begin production of its in-house AI chip, codenamed Iris, this September. The goal is to push total compute capacity to fourteen gigawatts by 2027. Two different companies sharing one constraint: you can't keep packing GPUs into a rack if the wires between them are electrical copper.

00:00:48 As Nvidia senior manager Jesse Clayton put it in IEEE Spectrum today, higher signaling frequencies increase attenuation, power draw, and heat. To cope, copper cables must become shorter and thicker, which makes routing through dense server racks increasingly difficult.

00:01:07 Engineers call this the "copper wall." With Nvidia planning racks that span up to five hundred seventy-six GPUs by 2027, the limit is no longer theoretical. The answer moving inside the rack isn't new in concept. Optical scale-up has been discussed for years. But the manufacturing side finally caught up.

00:01:29 Ayar Labs built optical chiplets meant to sit beside GPUs, converting electrical signals into light within inches of the compute silicon. Lightmatter took a different path: a photonic interposer that serves as the packaging substrate itself, stacking processors directly on top of a silicon photonics engine.

00:01:51 The remaining hurdle is laser integration. Current pluggable laser modules take up valuable rack space and don't scale well. Teams working on this are moving toward integrating lasers directly onto silicon to support much denser optical fabrics inside next-generation AI systems.

00:02:10 Nvidia's approach here treats Fusion as an umbrella rather than a commitment. You can put copper in it. You can put photonics in it. The ecosystem evolves side by side while the standard holds together. For infrastructure teams designing rack layouts or negotiating hardware procurement cycles right now, assume optical interconnects will start appearing in scale-up configurations within the next two years.

00:02:39 It's not a matter of if, only which integration path your preferred vendor picks first.

00:02:45 While hardware teams are thinking about rack space, the modeling layer is quietly shifting away from its core abstraction. A startup called Fundamental came out of stealth in February with a model named NEXUS. Unlike large language models that predict the next token in a linear sequence, NEXUS is a large tabular model.

00:03:08 It jointly learns from each row's numerical value, what that entry represents semantically, and how it relates to every other column in the table. The motivation is straightforward. Most enterprise data lives in spreadsheets. Banks use them for transaction logs.

00:03:26 Marketing agencies track web metrics. Clinical trials log vital signs. Large language models handle text and images fine until you feed them a structured table larger than a few rows, at which point they fall apart because token prediction requires sequential dependency that tabular data doesn't have.

00:03:47 Swapping columns changes nothing in the underlying facts, but it completely breaks an LLM's output. Fundamental pre-trained NEXUS on billions of tables using proprietary datasets, open-source corpora, and augmentation techniques to expand coverage across domains as different as biology and finance.

00:04:08 The result is a model that doesn't need months of bespoke feature engineering for each new use case the way gradient-boosted decision trees do. Amazon Web Services embedded NEXUS into SageMaker in June. That's meaningful because SageMaker is effectively the default operating system for secure machine learning.

00:04:30 Fundamental's confidential computing architecture means they physically can't access customer data, let alone train on it. The model runs where the data lives. Google launched its own competitor, TabFM, trained entirely on synthetic datasets. Feedzai and Mastercard released proprietary variants focused on financial crime prevention.

00:04:54 Researchers have released frameworks like FlexTab and TabICL, alongside an incremental modeling method, over the past year. The direction here is clearest when you look at what companies actually do with data. Most people don't like running queries or cleaning spreadsheets manually.

00:05:14 Automating that work requires understanding structure, not just generating plausible text. Large tabular models close the gap between sequential language modeling and deterministic prediction. If your stack still routes structured data through an LLM pipeline for analysis, you're accepting that trade-off today rather than facing a permanent constraint.

00:05:39 Two separate teams published notes on how orchestrator models are handling cost this week. Elvis shared that Meta launched Muse Spark 1.1 through the new Meta Model API, hitting performance near Opus-4.8 or GPT-5.5 levels while supporting a one million token context window.

00:05:58 The computer-use capabilities let agents write scripts when automation is faster, click directly when interaction is simpler, and generate batches of actions at each step. Meanwhile, Perplexity released a research preview of an orchestrator model built on GLM 5.2, post-trained specifically for their Computer harness.

00:06:21 Aravind Srinivas summarized the approach: pair it with an advisor for task routing, and you get near-frontier performance at roughly thirty-four percent of Opus's cost. The escalation pattern here is becoming the standard commercial move. Keep daily agent work on a cheaper base model.

00:06:41 Only route to frontier-tier inference when the task actually requires that capability. You save money on the common cases and still hit quality thresholds on the edge cases. One million token context windows remove the need for aggressive chunking in long-document analysis and multi-step workflows.

00:07:02 That alone changes how you structure your prompt routing layer. Instead of splitting queries by topic or length, you can feed an entire workspace state into the model and let it decide which sections need escalation. The pricing math will matter most once both services move past preview.

00:07:22 But the architecture is already settling: orchestrator models with computer-use capabilities and configurable escalation thresholds are replacing monolithic single-model agent pipelines. If you're building agents right now, test a two-tier setup before committing to a single provider's pricing tier.

00:07:43 The cost delta compounds quickly when those agents run continuously.

00:07:48 One other figure stands out today. Ollama closed its Series B with sixty-five million dollars from Theory Ventures, bringing total funding to eighty-eight million. The company hit eight point nine million monthly developers across eighty-five percent of the Fortune 500.

00:08:06 Co-founders Jeff Morgan and Michael Chiang came from Docker and Kitematic, applying containerization lessons to open-weight AI deployment. Peter Fenton led Ollama's earlier round and joined the board. His argument tracks with what enterprises are already doing: every company with high inference expenses has a vital project pushing them toward open-weight models for daily work.

00:08:31 Closed models stay in the portfolio for specific high-difficulty tasks, but the volume goes elsewhere. The community pushback from last year about cloud services drifting from the free desktop product is understandable. Morgan and his team have kept the core offering unchanged, adding GPU time billing rather than token limits for larger models that simply won't fit on a laptop.

00:08:57 That's a reasonable evolution when state-of-the-art open models can suddenly handle agentic coding tasks and no longer run reliably on consumer hardware alone. Scale shifts things here. Eight point nine million developers using a single tool to discover, download, and run local models isn't a niche project anymore.

00:09:18 It's the distribution layer for open-weight inference becoming commercial infrastructure. The local pass on today's lineup shows three converging tracks: physical interconnects moving from copper to light inside racks, modeling architectures splitting along sequential versus tabular lines, and developer tooling reaching ten million users while pricing models settle around escalation and GPU time rather than tokens.

00:09:46 You can't fix the physics of a data center. You can adapt your routing layers before they become legacy. And you don't need to wait for consensus to start testing tabular-native pipelines for the structured work your team actually does. That's the local reading on today's infrastructure shift.

00:10:05 Seln Oriax.