Google and AWS Add Native PyTorch Support for TPU and Trainium

Google and AWS are making their AI chips easier to use from PyTorch, lowering the cost of trying alternatives to Nvidia GPUs. At PyTorch Conference North America 2026, Google open-sourced TorchTPU, while AWS put TorchNeuron into public beta. Both projects add native PyTorch backends, giving teams a more familiar route to hardware they might otherwise evaluate only after a framework change.
That familiarity matters because accelerator decisions involve more than chip specifications. Developers also have to account for software support, model changes, debugging and the effort needed to make workloads perform well. A backend can reduce friction at the start of that process, but it can't make different hardware behave identically. The numbers here are underwhelming as a promise of effortless portability: the first run may be easy, while production performance can still demand specialist work.
Google brings PyTorch directly to TPUs
TorchTPU, available on GitHub, follows a preview with select customers in April. PyTorch says Google began the project in mid-2025. Developed with Meta’s PyTorch team, the stack lets developers run existing workloads on Google TPUs without JAX in the runtime. For teams already using PyTorch, that removes a framework transition from the initial evaluation.
The practical appeal is a shorter path from an existing model to a different accelerator. PyTorch says migration can require only “trivial code changes.” Developers can use eager execution to debug models, then apply torch.compile to improve TPU use and reduce memory overhead. That progression gives engineers a familiar way to separate basic correctness checks from later performance work.
Still, a low-friction first run answers only a limited question: whether the workload can execute. It doesn't establish how it will behave under production conditions, where throughput, memory use and distributed execution can shape the economics of a deployment. The distinction matters for teams comparing hardware on the basis of a working demo.
Google is pairing the backend with a broader set of ecosystem work intended to support PyTorch on Google Cloud and on-premises infrastructure. Announced integrations include Hugging Face, vLLM, SGLang, llm-d, TorchTitan, Ray and TorchRec. Google is also contributing LiteRT to the PyTorch ecosystem. The breadth of these connections signals that the project is meant to fit into surrounding tooling, not just launch a model on a TPU.
TorchNeuron targets Trainium instances
AWS is applying a similar approach to its own chips. TorchNeuron, released as open source and now in public beta on GitHub, supports Trainium2 and Trainium3 instances. The public beta gives developers a way to try that support, while also making clear the software is still being presented as a beta release.
For eager execution, AWS says developers can move a model to Trainium by changing its device call to .to(‘neuron’). Compiled workloads also need the Neuron backend selected, either as the default or within a torch.compile call. Those steps lower the barrier to an initial trial, though AWS qualifies that most teams can test Trainium through configuration changes alone; some workloads may need more adaptation.
That qualification is useful. Model behavior depends on the operations and execution patterns in use, so a small configuration change for one workload doesn't guarantee the same for another. If you're working in this space, treat a successful device switch as the start of an evaluation: it shows the path is open, not that the model is ready for a demanding service.
TorchNeuron also connects PyTorch with the Neuron Kernel Interface (NKI), a low-level language and compiler for AWS Trainium and Inferentia NeuronDevices. Developers can use it to write custom operators and tune performance. AWS and PyTorch have published guidance on execution modes and their tradeoffs. That low-level option matters because a native backend can ease access, while custom kernels give teams another way to address gaps between a portable model and chip-specific performance.
Portability comes before peak performance
These backends make hardware choice less tightly coupled to a framework rewrite, but a model that runs isn't necessarily a model that runs well. TorchTPU’s default Skinny Collectives approach illustrates the gap: for distributed workloads, PyTorch estimates it can be roughly 30–40% slower end to end than an optimized approach because it prevents the XLA compiler from overlapping computation with communication.
That example is a reminder that performance depends on how work moves through the system, not just on whether the model's operators are supported. Distributed workloads add communication between devices to the computation itself. When the software can't overlap those activities, the result can be slower execution even though the model is running successfully.
For production workloads that need maximum throughput, PyTorch recommends additional tuning, including Pallas kernels and stacks such as vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan-TorchTPU. The simple migration path is useful for evaluation; it doesn't remove the need to optimize communication and execution for the target hardware. Here's the thing: ease of entry and peak performance are separate promises, and these projects chiefly make the first one easier.
AWS’s own Trainium2 example shows what that work can involve. In a case study, AWS’s Generative AI Innovation Center worked with AI research and technology company i14 to adapt its Laplace 3B model. The team made three changes: it implemented the GLA attention kernel in native PyTorch, redesigned Muon communication and compiled the sequential backward pass. The example puts concrete engineering tasks behind the broader claim that some workloads require more than configuration changes.
Specialized tuning can be a substantial part of an accelerator evaluation. Teams need to weigh the work of adapting a model against the benefit they expect from a different chip, and that calculation will vary with the workload and the existing stack. A backend makes the comparison more accessible, but it doesn't settle it.
The next test is production value
Google’s open-source release and AWS’s beta give PyTorch teams more ways to test accelerators outside Nvidia’s CUDA ecosystem. The immediate gain is choice and a simpler first run; the next measure is how much chip-specific tuning teams need to make real workloads competitive. This is more significant than it looks: familiar APIs can bring more teams to an evaluation, but the software around a chip still shapes whether that evaluation leads anywhere.
For now, the practical question is how much of a team's existing model and tooling carries over, and what work remains after it runs. The answer will come from workload-specific results, including the effort required to tune execution and communication. More options are useful. The engineering cost behind them will determine their value.