Tool

Ollama

Local LLMs · Desktop + CLI · ollama.com

A local runner for open-weight language models. You pull a model, it serves an API on your machine, and you talk to it without sending every prompt to someone else's cloud.

What it is

Ollama packages model weights, a runtime, and a simple CLI/API so you can run models like Llama, Mistral, Gemma, and others on a Mac, Linux box, or Windows machine. It is the “docker-ish” path for local LLMs: ollama pull, ollama run, done. There is also a desktop app. Under the hood it is still your GPU/CPU, your RAM, your disk.

What it replaces

For a lot of everyday drafting, summarizing, and coding assistance, it replaces a paid hosted chat API (ChatGPT / Claude / Gemini subscriptions) when you do not need the absolute frontier model and you do want the data to stay on the machine. It also replaces rolling your own llama.cpp / vLLM stack if you just want something that starts.

It does not replace a frontier API when you need the strongest reasoning, the largest context, or tools that only exist in a vendor’s cloud. And it does not replace a properly sized inference server if you are serving a team.

License

The Ollama application (the runner / CLI / app) is released under the MIT License. That covers the software that downloads and serves models — not the models themselves.

Model licenses vary. Each weight file you pull carries its own terms (Meta Llama, Apache-2.0, Gemma terms, Qwen terms, etc.). Always read the model card before you ship anything commercial or redistribute weights. MIT on the runner ≠ free-for-anything on every model in the library.

Where it falls over

• VRAM and RAM are the real product. A 7B Q4 model is pleasant; a 70B model on a laptop is a pretend.
• Quality tracks the open weights you choose. Local ≠ smarter than the best closed API.
• Tool use, browsing, and multi-step agents are thinner than the polished hosted products unless you build the glue.
• Default setup is local-only; expose the API to a network without thinking and you have handed strangers a model endpoint.
• Some model licenses restrict use cases (e.g. certain commercial or high-user-count limits). The app will still download them.

Who it's for

Developers and power users who want private, offline, or cheap iteration on open models. People who are fine managing disk and GPU instead of paying a monthly API bill. Not a one-click “replace my job with AI” product — and that is the point.

Hardware that usually matters

The apps on these pages are free or free-to-use. The recurring cost is usually the machine: VRAM first, then system RAM, then disk for weights. Example Amazon listings skewed toward entry and mid budgets for beginners checking fit — not endorsements; stock and price move constantly.

• GIGABYTE GeForce RTX 3060 Eagle OC 12GB — entry 12 GB VRAM pattern many beginners still use for 7B–13B quantized chat before spending mid-range money.
• GIGABYTE GeForce RTX 4060 Ti AERO OC 16GB — mid 16 GB VRAM path when 12 GB starts feeling tight; still not an honest “70B fully on-GPU” card.
• EVGA GeForce RTX 3090 XC3 Ultra 24GB (Renewed) — older 24 GB class for larger local models / less painful offload; renewed stock and warranties differ from new.
• CORSAIR Vengeance LPX DDR4 32GB (2×16GB) — cheaper system RAM path for older DDR4 boxes when layers spill or you CPU-offload.
• Patriot Viper Venom DDR5 32GB (2×16GB) — DDR5 kit for newer boards; 32 GB is a common “I might spill layers” starting point.
• Samsung 990 EVO Plus 1TB NVMe SSD — entry NVMe for a first model library; step up when pulls fill the drive.
• Samsung 990 PRO 2TB NVMe SSD — default library disk for a mixed set of 7B/13B quants without deleting every Tuesday.

Some links are Amazon Associates. We may earn a commission if you buy.

← Tools · Home · ollama.com