Local LLM Guides

Where to start

If you're new to running local LLMs, work through these in order:

1. Install Ollama — the easiest way to run any open-weight model on Mac, Windows, or Linux. See our complete installation guide for every platform.
2. Pick a model — start with the rankings. Best Local LLMs for the overall top 8, or Best for Coding if you mainly want help writing code.
3. Match your hardware — our cost calculator tells you the 5-year total cost of ownership for buying hardware vs paying for API subscriptions, and the break-even point for each.
4. Choose your tools — decide whether you need a runtime, interface, serving engine, or knowledge layer. Browse the local AI tool library for the right fit.

In-depth guides (coming soon)

We're actively writing long-form, end-to-end guides for the most-requested topics. Subscribe to the blog to be notified when each is published.

• →Fine-tuning a local LLM on your own writing style — using QLoRA + Unsloth, 4 GB VRAM minimum.
• →RAG over a personal knowledge base — Ollama + ChromaDB + Open WebUI, 100% offline.
• →Speculative decoding on Apple Silicon — 2× faster inference with MTP on M-series Macs.
• →Multi-node inference with Exo — pooling 4 Raspberry Pis to run a 70B model.
• →Production deployment with vLLM — serving Gemma 4 31B at 200 tokens/second per user.