Definition
Tensor Processing Unit is a Tensor Processing Unit (TPU) is a chip Google designed specifically for the matrix math inside neural networks, built as an alternative to the general-purpose GPU for training and running AI models.
A Tensor Processing Unit, or TPU, is a custom AI chip designed by Google. Where a GPU started life as a graphics chip and was later adapted to AI, a TPU was built for neural networks from the start.
Almost everything a neural network does comes down to multiplying large grids of numbers, called matrices, and adding up the results. A TPU devotes most of its silicon to exactly that job. Its core is a systolic array: a grid of small arithmetic units that pass data directly to their neighbors, so numbers flow through the chip without constantly being fetched from memory. Google's first TPU was described in a 2017 paper as running in its data centers since 2015.
Think of a kitchen built for one dish. A general-purpose chip is a full restaurant kitchen that can cook anything. A TPU is an assembly line that makes one thing very quickly and wastes little.
TPUs are linked into large groups called pods, so thousands of chips can work on one model together. Google uses them to train and serve its own models, and rents them to cloud customers. Newer generations split into versions tuned for training and for inference, the two jobs that make up the cost of AI.
Most AI software is written against Nvidia's CUDA platform, which is a big reason Nvidia GPUs dominate. A TPU uses different software tools, so moving a workload between the two takes real engineering. That is why more places to rent TPUs matters: it gives builders a second supplier of compute, and a credible second supplier can put pressure on prices.
The tradeoff is portability. Code tuned tightly for one kind of chip may not run well on another. If you only call a model through an API, the chip underneath is the provider's problem. If you train or host your own models, it is yours.
TPUs are one kind of AI accelerator. Others include Nvidia and AMD GPUs and custom chips built by cloud companies and labs. All of them depend on fast memory next to the processor, which is why high-bandwidth memory matters as much as the arithmetic units. The chip you pick affects speed, cost per token, and how large a model you can serve.
Is a TPU better than a GPU? Neither wins everywhere. It depends on the model, the software and the price you are offered. Can you buy one? Historically TPUs were rented from Google Cloud, though Google has begun offering them in other ways. Check the provider's current terms.
A GPU is a flexible chip that began in graphics and now runs many kinds of parallel work. A TPU is a Google-designed chip built mainly for the matrix math in neural networks.
Google designs them. Customers have mainly rented them through Google Cloud.
Oracle says Google's TPUs reach its cloud in 2027, the first hyperscale cloud outside Google to offer them, and The Information reports OpenAI plans to buy the chips too. Nvidia now has a real rival for the compute you rent.