Running the model yourself
A local LLM is a large language model running on your own PC, phone, or company server rather than through a cloud service. The spread of open-weight models has made this a realistic option for individuals.
What you gain
- Privacy: input never goes to an outside server, which suits confidential or personal material
- Cost: once the environment exists, usage is unmetered — no API charges or subscription
- Offline: it works without a network connection
- Freedom: you can swap models, fine-tune, and customize
What you give up
You are bound by your hardware, which caps the size of model you can run. A typical PC runs small to mid-size models comfortably, and those are a step behind frontier cloud models in capability. Larger models need a powerful GPU and plenty of memory.
Quantization and the development of small but capable models keep widening what is practical locally, and dedicated apps have lowered the setup barrier considerably.
Who it suits
It is compelling for organizations with data that cannot leave the building, for sites with limited connectivity, and for anyone who wants to learn by taking the model apart. If you simply want the best answer quality with no effort, cloud services are enough. Cloud for everyday work and local for confidential work is the practical compromise most people land on. Running a small model once is also an effective way to understand how these systems actually work.
Ollama and LM Studio are the common runtimes — install the app, pick a model, and you are running. It is easier than most people expect.
Privacy and cost have driven adoption in government and healthcare too. Confidential data does not have to mean giving up on AI; running it locally is a real option.