TorchTPU
Google released TorchTPU, an open-source native PyTorch backend that makes the TPU a torch.device so PyTorch code runs on Google Cloud TPUs with minimal changes.
TorchTPU adds a native torch.device("tpu") backend to PyTorch. Developers no longer need to rewrite code in JAX or use the older torch_xla. Code runs eagerly or compiles through torch.compile using XLA, and collectives go through torch.distributed. Google also launched vLLM and SGLang engines built on it and a TorchTitan TPU fork.
- Date
- Tuesday 20 October 2026
- Lab
- Kind
- infra
- Access
- open weights
Figures
| Measure | Value | Measured by |
|---|---|---|
| Lowest cost per million tokens (Qwen 3.5 on Ironwood TPU, InferenceX) | 2.01 cents for the vLLM and SGLang TorchTPU engines, per the Google blog | company |
| Kimi K3 (2.8T parameters) decode speed with Pallas megakernel on Ironwood | more than 700 tokens per second to a single user vLLM-TorchTPU, co-developed with Inferact | company |
Software is open source under Apache 2.0 (a code backend, not model weights). The GitHub page shows version 0.2.0 on PyPI.
Sources
- github.com/google-pytorch/torch_tpu
- developers.googleblog.com/announcing-new-vllm-and-sglang-inference-engines-on-tpu/
- developers.googleblog.com/integrating-torchtpu-into-torchtitan-and-optimizing-models-for-t
- techcrunch.com/2026/10/20/google-has-joined-amazon-in-attacking-nvidias-stronghold-with-ai
This record was checked and corrected against its sources on 20 October 2026. How we check