Local Artificial Intelligence: Why AI Is Becoming Less Dependent on the Cloud

Summarizing a long email, transcribing a voice message, or sorting notifications: devices can handle more and more of these tasks without sending anything to a server. Two development trends have made this possible. The first is specialized chips in phones and laptops; the second is small language models that fit into limited memory. However, calling this “the end of the cloud” would be an exaggeration. Rather, the division of labor between the device and the data center is changing.

What Is Local Artificial Intelligence and Why Have Small Models Made It Possible?

Local artificial intelligence (known in technical terms as edge AI or on-device AI) means that the model runs directly on the user’s device. The input is processed on a phone or laptop, and the result is generated without going through the internet. The device therefore does not need a connection, and the response comes without any network-induced delay.

LLM vs. SLM: What a Small Model Is Good For

Large language models (LLMs) have hundreds of billions of parameters and run in data centers. Small language models (SLMs) typically have approximately 1 to 10 billion parameters. They do not attempt to know everything. They are designed for a limited set of tasks, such as summarizing text, rewriting it, classifying it, or answering questions about on-screen content.

Summarizing an email or categorizing a notification does not require a model capable of complex, multi-step reasoning. Such a small model is typically significantly faster and cheaper to run. However, it has an obvious limitation: it is less versatile than the largest models in the cloud.

How the Model Fits into a Phone

In addition to training models to be small from the start, compression is also used. For example, quantization reduces the numerical precision of the model’s weights, thereby decreasing its size and memory requirements. This comes at the cost of a slight loss in quality, which must be tested for a specific task.

NPU: Why New Chips Have Neural Processing Units

An NPU (neural processing unit) is a part of a chip designed for matrix calculations, which are the foundation of neural networks. It performs these operations with lower power consumption than a general-purpose processor. Performance is measured in TOPS, or trillions of operations per second.

This is significantly changing the market. According to an estimate by Counterpoint Research, phones capable of running AI models locally are expected to account for approximately 45% of global smartphone shipments in 2026, up from 36% in 2025. Gartner analysts define a “GenAI smartphone” as a device with an NPU capable of running small language models locally. According to their forecast, by 2027, an NPU with performance exceeding 40 TOPS will become the standard in premium phones.

What an NPU Can’t Do

TOPS figures are marketing-friendly, but they don’t tell the whole story on their own. The NPU is designed for small, long-running tasks and system-level AI functions, not for running massive models. In practice, software also complicates matters: popular tools for running models locally often do not utilize the NPU because they require models converted to a different format and compiled for a specific chip manufacturer. This is another reason why the performance listed in the table often doesn’t translate to real-world results.

Privacy: What Local Processing Guarantees—and What It Doesn’t

The main argument in favor of on-device AI is data protection. When an assistant reads messages, emails, and analyzes photos, it makes a difference whether this happens on the device or on a third-party server. For tasks processed exclusively locally, the data never leaves the device.

The reality, however, is more complex. Experts today consider the hybrid model to be the standard architecture. The local model handles routine tasks that are sensitive to latency and privacy, while complex or rare cases are offloaded to larger models in the cloud. For a specific feature, it is therefore worth determining when processing is offloaded from the device and whether the manufacturer discloses this information. Furthermore, the device itself becomes a target for attacks, so its security is just as important as that of a cloud-based solution.

Battery, Cooling, and RAM

Why RAM Requirements Are Increasing

The model must be loaded into memory while in use. Even a small model can take up around 1.8 GB, which is memory that is then unavailable to the rest of the system. According to one open-source AI developer, 6 to 8 GB of RAM is the “sweet spot” for keeping the model loaded alongside the system’s other functions.

Manufacturers are responding to this. According to available reports, some are restricting the most powerful on-device features to devices with at least 12 GB of memory, while 8 GB models are already below this threshold. For laptops labeled as “AI PCs,” some platform manufacturers have set a minimum of 16 GB. To claim that 8 GB is “definitely a thing of the past” would be an overstatement: basic functions still run on it. However, memory is becoming the threshold that separates basic AI from more capable AI.

Power and Heat

Even with an efficient NPU, long-term AI processing consumes power and generates heat—both of which are limited resources in a battery-powered device. That’s why local functions today are designed for short, quick tasks, not for continuous use. For users, this means that with intensive use, they should expect higher power consumption and the phone to heat up.

The Cloud Isn’t Going Away

According to forecasts by the consulting firm Deloitte, even in 2026, nearly all AI computing power will still be handled by data centers and powerful enterprise servers, not by phones and computers. The reason is simple: today’s NPUs are designed for lighter, one-off tasks, not for long, demanding computations. Local AI therefore complements the cloud rather than replacing it.

Conclusion

Small language models and NPUs don’t handle everything. However, they are sufficient for routine tasks where speed and privacy matter—and those tasks make up the majority of what we typically expect from a phone. For buyers, this leads to a practical rule: when choosing a device, in addition to TOPS, pay particular attention to memory size and which functions run locally and which run via the cloud. The most accurate picture so far is provided by a hybrid model, in which the device and the data center share the workload.