We plan to deploy disaggregated inference broadly across hundreds of megawatts in the Cerebras Inference Cloud and in partner deployments by the end of 2027. Our aim is to increase the ultrafast inference capacity available for our customers.
Cerebras disaggregated inference for infrastructure partners
The Cerebras disaggregated inference solution was also built for our partners to adopt. Infrastructure partners can deploy disaggregated inference clusters based on Cerebras-provided reference architecture, co-locating their chosen prefill accelerators and Cerebras systems within a unified network. For software integration, partners can either integrate with the Cerebras end-to-end disaggregated software stack or a decode-only SDK.
The end-to-end disaggregated software stack is best suited for partners optimizing time-to-market. It consists of:
- A disaggregated inference framework to route requests between prefill and decode pools and manage replica capacity and health.
- A prefill engine to process prompts using open-source inference engines like vLLM or SGLang.
- A Cerebras decode engine for response generation on Cerebras systems.
- KV Portal software, an interface to transfer intermediate model state, including key-value caches, between prefill and decode engines over RDMA.
The decode-only SDK is best suited for partners optimizing for control. With the decode-only SDK, partners can bring their own inference framework and prefill engine to the Cerebras decode engine and KV Portal software. In addition, the decode-only SDK provides development aids such as a mock Cerebras decode replica and a reference integration to an open-source disaggregated inference framework and prefill engine.