Definition

What is Neural processing unit?

Neural processing unit is a neural processing unit (NPU) is a chip, or a section of a chip, built to run the math of AI models efficiently on a phone or laptop, using less power than a general-purpose processor or graphics chip.

A neural processing unit, or NPU, is a processor designed for one job: the repetitive multiplication and addition that neural networks use. Phones, laptops and tablets now include one so they can run AI features without draining the battery or sending your data to a server.

How it differs from a CPU and a GPU

A CPU is a generalist. It handles almost any task, one careful step at a time, across a few powerful cores. A GPU does thousands of simple calculations at once, which suits graphics and AI training. An NPU goes further in specializing. It is built around the matrix math at the center of AI, and it gives up flexibility to get far more work per watt.

Think of three kitchens. The CPU is a chef who can cook anything. The GPU is a large crew making the same dish in bulk. The NPU is a machine that makes one dish extremely well on very little electricity.

What an NPU is used for

NPUs power features that need to run often and quickly on your own device: transcription, background blur on video calls, photo enhancement, text suggestions and small language models. This is the hardware side of on-device AI. Running locally keeps data on the device, works without a connection and avoids the cost of a cloud call for each request.

Different companies use different names. Apple calls its version the Neural Engine, Google's phone chips include a TPU-derived block, and many PC chips now advertise an NPU with a speed rating in TOPS, meaning trillions of operations per second.

Limits to know about

A TOPS number is a peak figure for a specific type of math, usually low-precision integers, so it does not tell you how fast a real model will run. Large language models are often limited by memory bandwidth and capacity, not raw compute. That is why a chip's memory size and design, such as unified memory, can matter more than its NPU rating when you run a big model locally. Models also usually need to be quantized to fit and run well on an NPU.

What to look for as a buyer

If you want local AI, check three things: how much memory the device has, whether the software you use can actually target the NPU, and independent speed tests in tokens per second for the model size you care about. A marketing number alone will not answer those questions.

An NPU is the dedicated cousin of the Tensor Processing Unit, which Google built for data centers. Both trade generality for efficiency, and both exist because inference is now a workload large enough to deserve its own silicon.

Questions people ask

What does an NPU do?

It runs AI model calculations on a phone or laptop efficiently, powering features like transcription, photo effects and small language models without much battery drain.

Is an NPU better than a GPU for AI?

It is more power-efficient for small, frequent on-device tasks, while a GPU is stronger for heavy workloads like training and running large models.