Challenging Nvidia's Dominance
Amazon Web Services has released a tool called TorchNeuron that lets AI startups easily move their AI apps onto Amazon’s AI chip, Trainium. In doing so, Amazon is also taking a direct stab at the software advantage that has helped Nvidia become the default hardware choice for AI apps and spurred its rise as AI hardware provider of choice.
TorchNeuron works with PyTorch, a free, open source toolkit that developers use to build, train, and run AI models. It lets AI apps written with PyTorch run on Trainium by changing just one line of configuration, according to Amazon — no code rewrite required.
Amazon says about 85% of AI apps are written with PyTorch. Facebook originally developed it, but since 2022, it has been part of the Linux Foundation, which maintains open source projects. Nvidia has spent years ensuring its CUDA software seamlessly supports AI apps written with PyTorch. As a result, Nvidia’s hardware is now deeply embedded in the toolkit, essentially by default.
Integration and Ecosystem
TorchNeuron is adapted from PyTorch, so developers who know PyTorch already know how to use it. Amazon isn’t the first company to attempt to easily bring their chips to the platform. PyTorch users can also port their apps to AMD GPUs or even Apple’s chips with a simple configuration change.
However, Amazon argues that porting to Trainium is not only easy, but it also gives AI startups access to Amazon’s entire cloud. AWS claims that this is a better choice for startups than using neoclods, many of which are tied to Nvidia and its hardware, like CoreWeave, Nebius, and Firmus.
Colin Brace, VP of Technology Software at AWS Annapurna Labs, noted that building on AWS gives founders the complete infrastructure needed to go from prototype to production seamlessly without relying on a single-purpose compute provider.
Price and Performance
AWS says that because the same chip both handles training and inference, and is tightly integrated with the cloud infrastructure that startups need, its cloud will save them money, even if AWS doesn’t offer the cheapest per-token price.
Trainium’s value lies in price-performance and unified training and inference execution. For instance, AI voice app provider Deepgram moved to Trainium using native PyTorch support and is processing 70% more audio per dollar at production-grade accuracy.
Upstream Contributions
Amazon is also pushing PyTorch to be less automatically centered on Nvidia, and it will contribute the code to do that to the main PyTorch project.
In the blog post announcing the new tool, the company outlined its broader strategy.
Amazon’s approach extends beyond its own hardware. The company is contributing changes upstream to PyTorch itself, working with the PyTorch community to remove code that ties the framework to specific chips. The goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users.
Industry Implications
While such a tactic benefits those selling hardware and cloud services that compete with Nvidia, PyTorch's primary goal is to simplify running AI code on hardware. Whether pushing Nvidia out gives developers more choice and affordable options or simply introduces configuration headaches remains to be seen.
If Amazon succeeds in uncoupling PyTorch from Nvidia, it could have significant ramifications for the GPU giant by removing the lock-in that kept developers from having alternative choices.
