Quantising Qwen3-32B by hand – a 32B model on 16 GB of VRAM
How I quantised Qwen3-32B with llama.cpp and an importance matrix so it runs on a 16 GB GPU: GGUF conversion, imatrix, quant levels compared, VRAM budgeting and layer offload.
How I quantised Qwen3-32B with llama.cpp and an importance matrix so it runs on a 16 GB GPU: GGUF conversion, imatrix, quant levels compared, VRAM budgeting and layer offload.
