Newshot

Overseas compute access narrows: Zhongzhi FlagOS and Qingmao Intelligence help domestic chips deliver commercial Token Factory at high performance

AI summary

On November 16, the US Bureau of Industry and Security (BIS) officially implemented its latest export control policy, extending restrictions from physical chips to remote compute access in an effort to block Chinese companies from obtaining advanced AI compute through overseas data centers. This external variable is rapidly transmitting to China's domestic compute supply side. On November 17, at the 2026 Qingmao AI Partner Conference and Token Factory Annual Meeting under the theme "Domestic Chips, Domestic Models, Independent Ecosystem," the Beijing Academy of Artificial Intelligence and Qingmao Intelligence jointly released an end-to-end inference optimization solution for a domestic Token Factory, officially launching a commercial Token Factory based on domestic chips built on Zhongzhi FlagOS, an open-source unified AI system software stack for multiple chips.

Translation(model translation, not the source's own text)

Overseas compute channels narrow! Zhongzhi FlagOS × Qingmao Intelligence help domestic chips launch high-performance commercial Token factories

On November 16, the latest export control policy of the U.S. Department of Commerce's Bureau of Industry and Security (BIS) officially took effect, extending restrictions from physical chips to remote compute access, aiming to block Chinese enterprises from obtaining advanced AI compute through overseas data centers.

This external variable is rapidly transmitting to the domestic compute supply side. For the emerging "domestic-chip Token factories," the narrowing of overseas compute channels is forcing the demand side to accelerate local compute's assumption of real business; but on the supply side, domestic inference chips also face stricter scrutiny in manufacturing and packaging. Hardware supply remains tight, and the industry has to confront a more urgent engineering proposition: on top of existing new domestic cards and massive legacy clusters, how can every card be squeezed for higher, more stable, and economically viable "effective Token capacity"?

On November 17, at the "Domestic Chips, Domestic Models, Autonomous Ecosystem" 2026 Qingmao AI Partner Conference and Token Factory Annual Meeting, the Beijing Academy of Artificial Intelligence and Qingmao Intelligence jointly released the "Domestic Token Factory End-to-End Inference Optimization Solution," officially launching a commercial Token factory based on domestic chips on top of Zhongzhi FlagOS (an open-source multi-chip unified AI system software stack).

Overseas compute channels narrow! Zhongzhi FlagOS × Qingmao Intelligence help domestic chips launch high-performance commercial Token factories
Overseas compute channels narrow! Zhongzhi FlagOS × Qingmao Intelligence help domestic chips launch high-performance commercial Token factories

The compute dilemma under the "scissors gap": electricity-price logic and ecosystem fragmentation

"In the first half of the game, large models have already proven that we can build world-class models. But in the second half, what matters more is how to turn the capabilities of these models into high-quality, low-cost, stably supplied, valuable Token capacity," 林咏华 pointed out in the opening speech. As large models enter the frontline industrial measurement system in the form of "Token capacity" for the first time, two real contradictions are coming to the fore.

The first is the "cyclical scissors gap" between model iteration and hardware infrastructure. Today's mainstream large models update their architectures almost monthly or even weekly, but underlying compute infrastructure is typically refreshed only every two to three years. If the software layer cannot keep up, expensive hardware will soon face the awkward situation of being unable to run new models shortly after deployment.

The second is compute cost and supply chain security. 林咏华 used the "electricity price" of the industrial era as an analogy: the competitiveness of Chinese manufacturing largely stems from low electricity prices, and the confidence in power supply comes from an energy structure that relies more than 90% on domestic coal; extrapolating to the intelligence era, the industry hopes for an affordable per-million-Token price, and the confidence in supply chain security depends on whether the underlying compute supply can take root domestically.

But the hardware side purchases by card and by scale, while the application side wants Tokens delivered by the second. As Qingmao Intelligence founder and CEO 关超宇 said at the conference, an efficient "translator" is urgently needed between the two to organize idle, heterogeneous, and hard-to-use domestic compute into measurable, highly reliable standardized capacity.

"Whether in the CPU era or the AI chip era, it is impossible for a single chip to dominate everything," 林咏华 believes. China does not lack excellent chip manufacturers; what is truly scarce is a software-hardware collaborative ecosystem that all hardware can share. In his subsequent sharing, Li Yan also pinpointed the engineering team's immediate pain points: facing a hundred flowers blooming of models and chips, the industry cannot start from scratch doing underlying adaptation and operator rewriting every time a model is released or a chip is launched.

This is exactly the original intent behind Zhiyuan's four-year continuous effort with universities and chip manufacturers to advance open-source FlagOS—using an open, unified, reusable system software stack to let the same set of operators and compilation systems share optimization experience across different hardware. To date, the FlagOS ecosystem has gathered 21 chip companies and 86 ecosystem partners, covering 35 AI chips. Its foundation consists of the general operator library FlagGems, the training and inference parallel framework FlagScale, the unified compiler FlagTree, and the unified communication library FlagCX. The nine major operator libraries have accumulated more than 1,454 open-source Triton operators, of which nearly 800 operators have achieved more than 80% of native operator performance on at least 5 domestic chips. At the framework level, Zhiyuan insists on not forking upstream code repositories, but instead uses a plug-in mechanism for zero-intrusion access to mainstream community ecosystems such as vLLM, SGLang, and Megatron-LM, achieving coverage of 20 vendors' backends with a single code repository, and over the past six months has repeatedly completed same-day (Day0) validation of up to 12 chips upon new model releases.

Deep into microsecond-level kernels: Zhongzhi FlagOS runs new domestic cards into commercial production lines

For Token factories pursuing commercial profitability, a model "running successfully" is only the passing line; whether latency can be lowered and throughput raised is what determines service quality and profit space.

In the jointly released "Domestic Token Factory End-to-End Inference Optimization Solution," the Zhiyuan team pushed optimization directly down to the microsecond-level kernels of Huawei Ascend 910C clusters, focusing on the Zhipu GLM-5.3 and GLM-5.3-Flash models. Li Yan introduced that large model inference has completely different bottlenecks in the compute-intensive Prefill stage and the memory-access-intensive Decode stage, and targeted remedies are necessary.

In the Prefill stage, targeting the hybrid attention architecture of GLM-5.3-Flash, the team restructured the large-block and small-block computation paths, coordinating the parallel division of labor between vector and matrix computation units inside the chip, eliminating frequent off-chip transfer of intermediate results. In tests with an input length of 81,920 and 16 concurrency, wave throughput improved by 3.41%; under 256K ultra-long input, TTFT p50 decreased by 4.94%.

In the Decode stage, most constrained by memory bandwidth, the team deeply fused fragmented operators—through LI quantized write-back technology, combining the originally stepwise dynamic quantization, scale conversion, and HBM cache write-back into the same kernel. In the 910C microbenchmark, this transformation compressed the original native pipeline latency of 18.03 microseconds to 2.04 microseconds, achieving nearly a 9x speedup; combined with vector-path customization for small-batch input, in end-to-end commercial scenario tests, Decode-stage ITL dropped by more than 10% overall, and TPOT decreased by more than 8%.

This performance dividend, released purely through underlying kernel reconstruction without relying on stacking hardware, is helping Qingmao Intelligence's Token factory handle higher concurrent commercial traffic at the same compute scale.

Revitalizing legacy 910B: letting old cards carry 1M long context

Compared with the performance sprint on the latest-generation chips, another more realistic challenge facing domestic compute centers lies in legacy assets. A large number of Ascend 910B servers have already been deployed in the market, but when facing new-generation models that routinely support million-token (1M) context, old cards often face a double crunch in memory and compute.

"Flash-class models like GLM-5.3-Flash and DeepSeek-V4.1-Flash are actually the chosen ones for domestic legacy cards." Li Yan offered a pragmatic judgment on site. Rather than waiting for a long hardware replacement cycle, it is better to use system software architecture to fully tap the potential of legacy 910B.

On Zhiyuan's internal BAAI Token Hub, the team used the FlagOS full stack to organize 48 legacy 910B cards into a "Prefill and Decode disaggregated" inference cluster, stably running GLM-5.3-Flash services supporting 1M context. 32 cards form 4 Prefill domains dedicated to digesting long-sequence input, while another 16 cards form 8 Decode domains focused on guaranteeing multi-concurrency streaming output. The service has run stably for two consecutive weeks, with online TTFT P50 maintained at around 2.3 seconds and TPOT P50 stable at 22 milliseconds.

A further efficiency leap comes from the reconstruction of the cache and transfer pipeline. On the framework plug-in side, the team enabled cross-Prefill-domain Prefix Cache reuse, split the cache into three tiers—local device memory, private host memory, and shared host memory—and combined segmented SDMA optimization to merge scattered memory pages into continuous transfer segments to raise PCIe bandwidth utilization. Measured data shows that this solution pushed the prefix hit rate from 82.65% to 94.15%, driving the median first-token latency in the reuse stage down from 16.63 seconds to 3.10 seconds, a drop of 81.4%.

This means that the large number of legacy 910B clusters in the hands of enterprises and research institutes need not be torn down and rebuilt; through software stack upgrades they can directly gain the ability to handle long-context commercial services. 林咏华 also revealed on site that, based on the FlagOS unified technology stack, Zhiyuan took only two weeks to complete adaptation and performance optimization of key large-model operators for Huawei's 910B-T chip.

Cross-chip horizontal validation and ecosystem "escort"

In Zhiyuan's view, the value of a unified software stack cannot be validated on a single chip alone. Cross-chip horizontal measurements announced at the summit show that on Pingtouge 810E, FlagOS 2.2 helped improve DeepSeek-V4.1-Flash inference performance by 77% to 82%, enabling 16-way concurrency availability, supporting up to 80 concurrent requests, and achieving a per-request decode speed of 136.2 tok/s; on Hygon BW1100, GLM-5.3-Flash inference performance also improved by more than 25%, with single-path throughput exceeding 80 TPS under multiple concurrency.

From the underlying operator library and compiler to cross-chip scheduling plug-ins, Zhiyuan's deep participation as one of the first strategic partners in this joint release is precisely intended to deliver the optimization capabilities accumulated in the open-source community directly to frontline Token factories and end enterprises.

As 林咏华 promised to industry partners at the end of the speech—Zhiyuan and the open-source community play exactly the role of "escort": "Whether what you have is a legacy cluster that has been running for years, a new-generation chip being deployed, or even a new-architecture hardware about to come out in the future, you don't need to worry about being unable to run the latest models on it." At a time when compute competition has shifted to competing over "effective Tokens," an open, reusable system software foundation is becoming the key fulcrum for domestic compute to break free from cyclical constraints.

  • Zhiyuan Wujie · PhysCapBench: the first cross-domain world model "physical ...
  • Learning by Doing with AI, Creating the Future | Zhiyuan SoulAgent continues to expand educational applications
  • First Yuanchuang Program roadshow: five AI hard-tech projects showcased, Zhiyuan ...
  • Zhiyuan Zhongzhi FlagOS 2.2 enables frontier models and diverse compute to evolve collaboratively

The translation is machine-generated and individual terms may be off. Nothing is rewritten or cut; the original is authoritative.

Published by 雷锋网
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗