Processing that stays on the device
On-device AI runs AI processing on a phone, laptop, or other device rather than on a server over the internet, using the CPU, GPU, or a dedicated neural accelerator.
It covers speech recognition, image classification, summarization, and smaller language models. Apple Intelligence handles much of the work involving personal information on the device itself, reaching for Private Cloud Compute only when a request needs more computation.
Not a vendor term
On-device AI is not an Apple product name. It is a general term that has been used in machine learning for years, so there is no single launch event that introduced it.
It drew wide attention as a way of explaining Apple’s AI strategy at WWDC24, alongside the Apple Intelligence announcement. WWDC26 widened the options further through Apple Foundation Models, the Foundation Models framework, Core AI, and MLX, making it more practical to run generative AI and agents locally.
Advantages
- Input can be processed without leaving the device
- It works where connectivity is poor, or offline
- No network round trip, so responses can be faster
- It avoids per-call cloud API costs
How it relates to local LLMs
A local LLM is a large language model running on your own hardware. On-device AI is the broader term, covering image recognition, audio processing, and predictive models as well as LLMs.
Running locally does not automatically mean safe, accurate, or fast. Model size is constrained by the device’s memory and power budget, and older hardware may not support a feature at all. Deciding which work belongs on the device and which belongs in the cloud is the real design question.