TPU Sync
Google Cloud released TPU Sync, an open-source C++ library for moving KV caches and model weights on Cloud TPUs, reporting over 90% lower KV transfer latency on large MoE models.
TPU Sync treats TPU HBM, host DRAM, shared storage and the data center network as one memory hierarchy. It has a native C++ core with Python wheels for PyTorch and JAX. It plugs into vLLM and SGLang as a KV connector and as a prefill/decode transfer and hierarchical cache backend.
- Date
- Friday 13 November 2026
- Lab
- Kind
- infra
- Access
- open weights
Figures
| Measure | Value | Measured by |
|---|---|---|
| KV transfer latency reduction | over 90% on 235B to 480B parameter MoE models; benchmarks used TPU v7x (2x2x1) | company |
Open-source library (code, not model weights) that runs on Cloud TPU v7x (Ironwood) and v6e.
Sources
This record was checked against its sources on 13 November 2026. How we check