This source only provides a headline — there is no body to show. Use “Read the original” below to read it at the source.
Google Releases TPU Sync: Open-Source KV Cache Transfer for TPUs
Google Cloud has released TPU Sync , an open-source C++ library for moving KV caches and model weights on Cloud TPUs. The Google TPU Sync KV cache transfer library treats TPU HBM, host DRAM, shared storage and the data center network as one memory hierarchy. It targets a growing bottleneck: moving model state, not computing it. Google reports over 90% lower KV transfer latency on 235B to 480B parameter MoE models. TL;DR Library: Native C++ core with Python wheels for PyTorch and JAX. Tested on 235B to 480B MoE models. Runs on: Cloud TPU v7x (Ironwood) and v6e per the README . Benchmarks used TPU v7x (2x2x1), PCIe Gen5 and 200 Gbps NICs. Performance: Large cuts to KV transfer latency and time to first token (TTFT) in Google’s internal lab tests. Best: Up to 90% lower average TTFT (480B P/D serving at low concurrency, and 64K-context prefix caching). Worst: No RL numbers published. The README says it is not yet recommended for general use. Bottom line: Best: Qwen3 235B KV transfer drops…