The Humanevi newsletter
Weekly
Once a week: 5 things worth your time.
1 story, 1 idea, 1 discovery, 1 tool, 1 book.
The Humanevi newsletter
Once a week: 5 things worth your time.
1 story, 1 idea, 1 discovery, 1 tool, 1 book.
A rough, rule-of-thumb guide to which local AI model sizes your hardware can realistically handle.
Large models
Roughly up to 22B parameters — near top-tier open models, best with a capable GPU.
These figures use widely cited rule-of-thumb approximations (~1GB per billion parameters at 8-bit, ~0.5GB per billion at 4-bit, with headroom reserved for context and the OS) — not precise specifications. Actual usable size depends on the model architecture, context length, quantization method and software you use.
This tool is for people who want to run an AI model on their own computer instead of the cloud and need to know how large a model their hardware can handle. Enter your system RAM and, if you have one, your graphics card VRAM, and it shows the range of local models that will likely fit.
The calculator relies on two common rules of thumb: roughly 1 GB of memory per billion parameters at 8-bit quantization, and about 0.5 GB at 4-bit. First it holds back 30% of your memory for the operating system, the inference software and the context window, so only 70% counts. The output is the maximum billions of parameters at 8-bit and at 4-bit, plus a size tier (small, medium, large and so on) based on the 4-bit figure.
With 16 GB of RAM, 11.2 GB is usable (70%). At 8-bit that works out to about 11.2 billion parameters; at 4-bit, about 22.4 billion. Since 22.4 falls between 13 and 34, the tool labels it as large models. With 32 GB the numbers double to 22.4 and 44.8 billion, which lands in the very large tier.
This is a rule-of-thumb approximation, not a spec sheet. Real capacity depends on the model architecture, context length, the software you use and the speed you expect. Running on a CPU works but can be quite slow.
