System Requirements Guide

System requirements for local AI come down to three things: model size (measured in parameters), quantization (shrinking a model by storing its numbers at lower precision) level (how compressed the model is), and whether you have a GPU or are running on CPU alone. This guide gives you specific, up-to-date numbers for every setup level. Whether you are a beginner with a basic laptop running Gemma 4 E2B or an advanced user running Llama 3.3 70B on a professional workstation, you will find a clear answer here.

“The fundamental rule: VRAM ≈ 2 GB per 1 billion parameters at full precision (FP16). Q4 quantization quarters that. An 8B model at Q4 needs roughly 4.8 GB VRAM, which fits comfortably on any modern gaming GPU.”

• 1RAM vs VRAM: what is the difference? - RAM (System Memory) is your computer's main memory. All local AI tools can run models using RAM alone, without a GPU. This is called CPU-only inference. It works, but it is significantly slower. VRAM (Video RAM) is the memory on your GPU (graphics card). When a model fits entirely in VRAM, generation is fast, often 10–50x faster than CPU-only. VRAM is the single biggest bottleneck for running local AI efficiently.
• 2Quick VRAM estimation formula - A practical rule of thumb for Q4 quantized models: VRAM needed (GB) ≈ Number of billions of parameters × 0.6. Examples: 8B model → 8 × 0.6 = ~4.8 GB VRAM. 14B model → 14 × 0.6 = ~8.4 GB VRAM. 31B model → 31 × 0.6 = ~18.6 GB VRAM. 70B model → 70 × 0.6 = ~42 GB VRAM. For MoE models: use active parameters, not total.
• 3Q4_K_M is the recommended starting point for most users - Q4_K_M gives you excellent quality at a size most modern hardware can handle. It reduces model size by about 70% compared to full precision, with only about 5% quality loss. Start here unless you have a specific reason to use a different quantization level.
• 4MoE models use active parameter count for VRAM sizing - Llama 4 Scout has 109B total parameters, but only 17B are active per query. So VRAM sizing is based on 17B (roughly 10–12 GB), not 109B. Always check the active parameter count when evaluating MoE models. Storage still reflects the full parameter count.

• 1Can I run local AI on a Mac? - Yes. Apple Silicon Macs (M1, M2, M3, M4) are among the best consumer hardware options for local AI. Their unified memory architecture means system RAM is shared with the GPU. An M3 Pro MacBook Pro with 18 GB unified memory can run 7B–14B models efficiently. Ollama and LM Studio both support Apple Silicon natively via Metal.
• 2What is the minimum setup I need to get started? - Any modern computer with 8 GB of RAM can run Gemma 4 E2B or Phi-4-mini on CPU-only using LM Studio or Ollama. You will get 5–15 tokens/second, which is slower than cloud AI, but fully private and functional. A GPU with 6+ GB VRAM will dramatically improve speed.
• 3Is CPU-only inference usable? - For small models (2B–7B at Q4), CPU-only inference is quite usable, especially on modern multi-core CPUs. Expect 3–20 tokens/second depending on your CPU and model size. For 13B+ models on CPU-only, generation becomes noticeably slow (1–5 tokens/second). A GPU is strongly recommended for models larger than 7B.
• 4What is the best GPU for local AI on a budget? - The NVIDIA RTX 3060 12 GB (~$280 used) is the standout budget pick. Its 12 GB VRAM fits 7B–13B models fully and handles most local AI workloads well. The RTX 4060 Ti 16 GB (~$450 new) is a solid step up for 14B models with room to spare. Avoid the RTX 4060 Ti 8 GB variant for serious local AI work, as the VRAM ceiling is too low.
• 5How much storage do I need? - A good starting point is 50 GB of free SSD space, which gives you room for 3–5 models at different sizes. Larger model collections at 70B+ can easily reach 200 GB. NVMe SSDs are recommended because model loading from NVMe is significantly faster than from SATA SSDs or spinning hard drives.