Newshot

Inference now drives two-thirds of AI workloads, changing where chips get deployed

Summary

AI spending is shifting from giant training clusters toward smaller, quickly built sites, and modular data centers are becoming a natural home for inference workloads. Power, cooling and memory supply now shape where tokens get generated as much as raw chip performance does. Inference-first chipmakers want to capture that shift. Positron AI Inc., which raised […] The post Inference now drives two-thirds of AI workloads, changing where chips get deployed appeared first on SiliconANGLE .

Original text(via SiliconANGLE)

UPDATED 13:28 EST / NOVEMBER 21 2026

Inference now drives two-thirds of AI workloads, changing where chips get deployed

by Kelly Knight

AI spending is shifting from giant training clusters toward smaller, quickly built sites, and modular data centers are becoming a natural home for inference workloads. Power, cooling and memory supply now shape where tokens get generated as much as raw chip performance does.

Inference-first chipmakers want to capture that shift. Positron AI Inc., which raised $875 million at a $5 billion valuation in September, builds inference appliances around commodity memory instead of scarce high-bandwidth memory. Inference has overtaken training as the force behind infrastructure buying decisions, according to Cameron McCaskill (pictured), head of sales and business development at Positron.

“We’re at a point where about two-thirds of the workloads being run through these AI infrastructures are inference versus one-third is training. And back in 2023, it was like 85% training, 15% inference,” McCaskill said. “So now you can see why inference is driving the economic decisions of how they’re going to build out their infrastructure.”

McCaskill spoke with theCUBE Research’s Rebecca Knight and John Furrier at SC26, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed inference overtaking training, Positron’s commodity-memory chip design and why low-power modular data centers suit inference. (* Disclosure below.)

Why modular data centers suit low-power inference

Positron’s first-generation Atlas server ran on a programmable chip, and the company plans to tape out its own silicon early next year for its Titan server. Oracle Corp. is bringing about 340 Atlas servers live in its cloud this quarter, McCaskill noted. A neocloud provider has also signed on for Atlas and committed to Titan. Positron’s central design choice was skipping high-bandwidth memory.

“Everyone else pretty much in the industry is using what’s called HBM or high bandwidth memory. It’s very limited in supply, very expensive,” McCaskill said. “We went with commodity memory.”

That choice also keeps power low enough for air-cooled rooms. A single Positron server draws about 4,500 watts, and a full rack of Titan servers fits within 40 kilowatts, McCaskill explained. That footprint suits sites where operators can find a few megawatts of cheap power without heavy regulatory hurdles. For inference providers monetizing new capacity, deployment speed is a big part of the appeal, according to McCaskill.

“One of the groups of companies that we’re seeing a rise in right now and coming to us for product is these modular data centers, where they have a half megawatt or, 1 megawatt or 2.5 megawatt modular data center, and those can be deployed very quickly,” McCaskill said. “And the goal of a lot of these inference players is I want to get from land and power to monetizable tokens within 90 to 120 days. And those modular data centers, coupled with our technology, can be a great solution.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of SC26:

(* Disclosure: NYSE Wired sponsored this segment of theCUBE. Neither NYSE Wired nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

  • 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
  • 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

  • Sovereign AI: Where control becomes an architectural requirement
  • CrowdStrike reportedly in talks to acquire cybersecurity provider Zscaler
  • Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium
  • Trump announces plan to tax US companies that build data centers abroad
  • Equifax cuts database firefighting after moving billing to Oracle on AWS
  • Sovereign AI: Where control becomes an architectural requirementAI - BY GUEST AUTHOR . 39 MINUTES AGO

Sovereign AI: Where control becomes an architectural requirement

AI - BY GUEST AUTHOR . 39 MINUTES AGO

  • CrowdStrike reportedly in talks to acquire cybersecurity provider ZscalerSECURITY - BY MARIA DEUTSCHER . 17 HOURS AGO

CrowdStrike reportedly in talks to acquire cybersecurity provider Zscaler

SECURITY - BY MARIA DEUTSCHER . 17 HOURS AGO

  • Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York SymposiumAI - BY VICTORIA GAYTON . 18 HOURS AGO

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

AI - BY VICTORIA GAYTON . 18 HOURS AGO

  • Trump announces plan to tax US companies that build data centers abroadPOLICY - BY MARIA DEUTSCHER . 18 HOURS AGO

Trump announces plan to tax US companies that build data centers abroad

POLICY - BY MARIA DEUTSCHER . 18 HOURS AGO

  • Anthropic researcher claims to have proven Riemann hypothesis using ClaudeAI - BY MARIA DEUTSCHER . 21 HOURS AGO

Anthropic researcher claims to have proven Riemann hypothesis using Claude

AI - BY MARIA DEUTSCHER . 21 HOURS AGO

  • Equifax cuts database firefighting after moving billing to Oracle on AWSCLOUD - BY JONATHAN ANTHONY . 22 HOURS AGO

Equifax cuts database firefighting after moving billing to Oracle on AWS

CLOUD - BY JONATHAN ANTHONY . 22 HOURS AGO

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by SiliconANGLE
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗