Quantization
Quantization is the process of storing neural network weights in lower bit widths (such as 4-bit or 8-bit) so a model uses less VRAM while staying usable for local inference.
Quantization maps high-precision floats to compact integer codes. Tools like llama.cpp, Ollama, and LM Studio expose GGUF quants (Q4_K_M, Q5_K_S, and similar) so you can pick a file that matches your GPU.
When you quantize aggressively, perplexity rises and long-context reasoning can wobble. For day-to-day chat on a 12 GB card, Q4 is often the sweet spot; for coding or math, Q5 or Q6 may be worth the extra gigabytes.