Local LLM Guides
Step-by-step walkthroughs for getting a local LLM running, optimized, and integrated into your daily workflow. Each guide is end-to-end — no "and then figure it out yourself" gaps.
Where to start
If you're new to running local LLMs, work through these in order:
- Install Ollama — the easiest way to run any open-weight model on Mac, Windows, or Linux. See our complete installation guide for every platform.
- Pick a model — start with the rankings. Best Local LLMs for the overall top 8, or Best for Coding if you mainly want help writing code.
- Match your hardware — our cost calculator tells you the 5-year total cost of ownership for buying hardware vs paying for API subscriptions, and the break-even point for each.
- Choose your tools — Ollama for the runtime, but you'll want a UI. Compare Ollama vs LM Studio vs Jan vs llama.cpp for the right fit.
In-depth guides (coming soon)
We're actively writing long-form, end-to-end guides for the most-requested topics. Subscribe to the blog to be notified when each is published.
- →Fine-tuning a local LLM on your own writing style — using QLoRA + Unsloth, 4 GB VRAM minimum.
- →RAG over a personal knowledge base — Ollama + ChromaDB + Open WebUI, 100% offline.
- →Speculative decoding on Apple Silicon — 2× faster inference with MTP on M-series Macs.
- →Multi-node inference with Exo — pooling 4 Raspberry Pis to run a 70B model.
- →Production deployment with vLLM — serving Gemma 4 31B at 200 tokens/second per user.