Multiple Choice Flashcards

Keep the same calibration set and tune serving batch size first. Use quantization-aware training or better calibration data. Compare compressed and original models using training accuracy alone. Try int4 because smaller weights are less noisy.