AI
Evaluating quantised models: perplexity, KL divergence & real task evals
How do I know IQ3_M is still good enough? Measuring the quality loss of a quantisation – with llama-perplexity, KL divergence against the base model, and a small task eval of your own.
• Alain Ritter