Step 1: inventory the machine
Record total system RAM, GPU model and VRAM, CPU, operating system and free storage. For local AI, VRAM often determines how much of a model can stay on the GPU, while system RAM determines how much room you have for CPU execution or offload.
If you do not know those numbers, Windows Task Manager can provide a basic view. A dedicated monitor such as HardwareRadar can organise the machine-level information more clearly.
Step 2: define the task before the model
Do you need fast chat, coding assistance, document summarisation, reasoning, vision or an agent planner? The best fit changes with the task. A smaller specialised model can be more useful than a larger general model that runs slowly.
Step 3: choose a realistic size band
Model parameter count is only a rough scale. Quantisation changes the stored size, and context/runtime overhead adds memory. For an initial shortlist, smaller quantised models make sense on limited hardware; systems with 8–12 GB of VRAM have more options; 16 GB and above provides substantially more headroom. These bands are guidance, not compatibility guarantees.
Read How Much VRAM Do You Need for Local AI? for the memory factors behind this estimate.
The easy version: four fit states
You do not need to memorise model maths. A useful model-fit result can be reduced to four states:
- Full GPU: the estimated model, context and runtime workspace fit inside a conservative dedicated-VRAM budget. This is normally the best place to start.
- Hybrid: it needs both GPU VRAM and system RAM. It may run, but latency can rise sharply.
- CPU / system RAM: it can fit without enough GPU memory, but interactive speed should never be assumed.
- Does not fit: the estimated working set exceeds the safe memory budget. Choose a smaller model, shorter context or more memory.
The important distinction is simple: memory fit is not a speed claim. A model can load successfully and still be the wrong choice for daily use.
Step 4: check the runtime
A model file fitting in memory does not guarantee that your selected runtime supports the architecture, quantisation, GPU backend or features you need. Ollama, LM Studio and other runtimes may expose different model catalogues and acceleration paths. Verify the exact model/runtime combination rather than assuming portability.
Step 5: distinguish “loads” from “works well”
CPU or partial GPU offload can make a model run on hardware that cannot hold it entirely in VRAM. That can be useful for batch work, but interactive speed may be poor. A practical test should include startup time, time to first response, sustained generation speed, memory pressure and task quality.
For a hands-on Windows setup and our measured 11 GB RTX workstation comparison, continue with How to Run a Local LLM on Windows in 2026.
A conservative selection pattern
- CPU-only / very low VRAM: prioritise small quantised models and modest context.
- 6–8 GB VRAM: start with smaller models and test task quality before moving upward.
- 10–12 GB VRAM: broaden the shortlist to medium-size quantised models where runtime support is proven.
- 16 GB+ VRAM: test larger models or longer contexts, but keep workload fit ahead of size.
- 24 GB+ VRAM: much greater flexibility, still bounded by architecture, context and performance goals.
Use measurements, not leaderboard assumptions
Public benchmarks are useful signals, but your local runtime, prompt type and hardware determine practical results. LLMRadar is built around that principle: detected hardware and a stated goal should drive recommendations, with local benchmarks used as evidence rather than an absolute universal ranking.
The Local AI Readiness Estimate now applies the same conservative sizing rules used by the ARKTOR model-fit core: model weights, selected context and runtime workspace are estimated separately, then the result is classified as Full GPU, Hybrid, CPU/system RAM or Does Not Fit. It remains browser-only and deliberately makes no tokens-per-second claim.