Skip to content

Cluster track · Presentation

Your AI Cluster's Real Bottleneck Is How Fast It Can Change

When
Wednesday, February 3, 202712:35 PM to 12:50 PM PT
Where
Room BSan Jose McEnery Convention Center
Track
ClusterArchitecting 1M GPU Clusters. Programmed by CloserStill with OCP
Format
Presentation

About this session

As large-scale AI clusters scale to tens of thousands of accelerators, hardware failures become an inevitable, regular occurrence rather than exceptional events. Consequently, the true bottleneck to maintaining high compute efficiency ("goodput") lies in the operational speed and safety of the release process, updating the complex, co-designed stack of firmware, drivers, run-times, and diagnostics across a disaggregated fleet while training jobs continue running.

This article examines the unique challenges of release engineering in AI data centers, where silent interactions between components, tight hardware-software dependencies, and high-power density can trigger fleet-wide performance degradation or job interruptions. Key strategies to mitigate these risks include establishing baseline-relative health metrics, subjecting rollbacks to rigorous stress testing on dedicated fleets, and leveraging AI coding agents for specific tasks like failure triage and workspace cleanup. Ultimately, while hardware design sets the theoretical ceiling for AI performance, operational success is determined by how quickly and safely teams can deploy changes across the underlying infrastructure.

General rate, until Feb 2, 2027

Register for Digital Infrastructure World 2027

One pass covers this session and every other session, plus OCP Tech Day, the keynotes and the exhibit hall, on all three days.

  • All Access$799
  • Open Compute Project members$399
  • Teams of 3 or more, each$599.25

Prices in US dollars, per person. The rate changes on the date shown.

More in the Cluster track

All sessions