Guides

In-depth guides: VRAM per billion parameters, quantization explained, best GPUs for local LLMs, Mac vs GPU, multi-GPU setups, and more.

• Best GPU for Local LLMs in 2026: Every Budget Tier

  The 2026 buyer's guide for local LLM hardware: best used value (RTX 3090), best new cards (RTX 5090), best big-memory option (Mac Studio), and what to skip.

  verified 2026-08-04

• CPU Offload: Is Partial GPU Offloading Worth It?

  llama.cpp -ngl explained: what partial offload does to speed (the 60-100 GB/s memory wall), when it's a fair trade, and when a smaller model wins.

  verified 2026-08-04

• How Much VRAM per Billion Parameters? (2026 Cheat Sheet)

  VRAM-per-parameter cheat sheet: exact GB per billion parameters at FP16, Q8, Q6, Q5, Q4, Q3 and Q2, plus KV cache and overhead — with worked examples.

  verified 2026-08-04

• KV Cache Explained: Why Long Contexts Eat Your VRAM

  What the KV cache is, the exact formula, why 128K context can cost more than the model itself, and how MLA and q8 KV cache change the math.

  verified 2026-08-04

• Mac vs GPU for LLM Inference: Unified Memory Explained

  Apple Silicon unified memory vs NVIDIA VRAM for local LLMs: real bandwidth numbers (273/546/819 GB/s), the 75% usable-memory rule, and who should buy which.

  verified 2026-08-04

• Multi-GPU Setup for Local LLMs: NVLink, PCIe & Layer Splitting

  How to run 70B+ models on multiple GPUs: tensor parallelism vs layer splitting, why you don't need NVLink for inference, and the real speed math.

  verified 2026-08-04

• Quantization Explained: Q4 vs Q8 and What You Actually Lose

  What GGUF quantization really costs: Q8_0 vs Q6_K vs Q5_K_M vs Q4_K_M vs Q3/Q2 — measured file sizes, quality guidance, when to go below Q4.

  verified 2026-08-04

• Running DeepSeek-R1 Locally: The Complete Hardware Guide

  DeepSeek-R1 671B needs ~413GB at Q4 — every realistic path: the Mac Studio truth, multi-GPU rigs, R1 distills that fit, and when to use the API.

  verified 2026-08-04