Guide
VPS vs local hardware for LLMs
“Just rent a GPU” and “just buy a 3090” are both incomplete answers. This page is the boring decision: when a box under your desk wins, when a VPS or cloud GPU is the sane path, and how each fails. Pair it with VRAM & RAM and storage for local models. Not a coupon page.
What this replaces
It replaces thumbnail certainty (“cloud is always cheaper” / “self-host everything”) with a few decision axes: how often you run models, how big they are, whether you need the weights offline, and whether you are also hosting the self-hosting stack (Immich, Nextcloud, Coolify, etc.).
It does not replace reading current cloud GPU price sheets, power costs in your region, or the license on the model you load. Numbers below are planning shapes, not quotes.
Local hardware — when it wins
- Daily chat / coding agents on mid-size quants. If you live in Ollama or LM Studio for hours a day, owning VRAM often beats metered cloud idle + round-trips.
- Privacy and offline. Weights and prompts stay on your LAN. No “is this tenant isolated?” debate for personal notes.
- Fixed cost psychology. You already paid for the card and the NVMe library. Marginal cost is electricity and fans.
Failure modes: under-buying VRAM (see VRAM bands), under-buying disk, noisy boxes in small apartments, and the classic “it loads on CPU offload so I bought the wrong card” trap. Laptops with shared memory do not inherit desktop VRAM charts cleanly.
VPS / cloud GPU — when it wins
- Burst or rare large models. Occasional 70B-class experiments can be cheaper to rent for a weekend than to buy a second used 24 GB card.
- Always-on small services without a home lab. A cheap CPU VPS for Coolify, reverse proxy, or light APIs is a different product than a GPU box — do not confuse them.
- Travel / thin client. Thin laptop + remote GPU beats shipping a tower.
Failure modes: GPU VMs that cost more per month than a mid-range card’s amortized year; cold-start and image pull delays; egress fees; providers that rename GPU SKUs every quarter; and “always-on” habits that erase the savings. CPU-only VPS droplets are not a substitute for local inference of anything non-trivial.
Honest cost shapes (not quotes)
- Local mid-range (e.g. 16 GB class GPU + 64 GB RAM + 2 TB NVMe): large one-time spend; then power. Fine if you use it most days.
- Used 24 GB class: still what many people hunt for larger local models; availability and PSU needs are the tax.
- CPU VPS (app hosting): often a few dollars to a few tens per month — great for the self-hosting plumbing, useless as a 13B chat GPU.
- Cloud GPU by the hour: can win for short experiments; loses when you leave the instance up “just in case.”
Example local hardware (disclosed Associates)
Same verified examples as the
VRAM guide and
local LLMs hub — categories only, not a ranking. Tag
techgurunomic-20.
- ASUS Prime GeForce RTX 5060 Ti 16GB — mid-range 16 GB VRAM pattern for many 7B–13B quantized setups.
- GIGABYTE GeForce RTX 3090 24GB — older generation; 24 GB still matters for larger local models or less painful offload.
- CORSAIR Vengeance DDR5 64GB (2×32GB) — system RAM for spill, Docker UIs, and browsers.
- Samsung 990 PRO 2TB NVMe SSD — room for a real model library.
Some links are Amazon Associates (tag=techgurunomic-20). We may earn a commission if you buy.
VPS providers (one affiliate link, disclosed below)
Official sites only — easy to swap later. Not endorsements. Compare current GPU vs CPU SKUs yourself.
- DigitalOcean — common for app/VPS hosting; check whether you need a GPU product or a plain droplet.
- Hostinger — VPS/hosting for sites and light services; not a substitute for local GPU inference.
- Contabo — low-cost, high-RAM CPU-only VPS plans with US and EU locations; fine for small quantized models or an always-on API/front-end, not a GPU box.
The Contabo link is an affiliate link; Tech-Gurunomics may earn a commission if you buy, at no extra cost to you. The DigitalOcean and Hostinger links are plain.
For deploying your own apps on a VPS you control, see Coolify and the self-hosting cluster.
Where the advice falls over
- Electricity, noise ordinances, and roommate vetoes do not show up on GPU charts.
- Team multi-user serving (vLLM-shaped) is a different purchase than solo chat.
- Apple Unified Memory and NVIDIA VRAM are not interchangeable shopping lists.
- Provider marketing “AI VPS” labels sometimes mean CPU with a marketing adjective.
Verdict
Buy local metal when you use mid-size models most days and care about offline. Rent when the workload is bursty, oversized, or you are traveling — and shut it down when idle. Host the always-on self-hosting stack on a boring CPU VPS if you lack a home lab. Size the box with VRAM & RAM; fill the library with storage notes; run software from the Local LLMs cluster.