ISTA-DASLab/Meta-Llama-3-8B-AQLM-2Bit-1x16

Official AQLM quantization of meta-llama/Meta-Llama-3-8B .

For this quantization, we used 1 codebook of 16 bits.

Results:

Model	Quantization	MMLU (5-shot)	ArcC	ArcE	Hellaswag	Winogrande	PiQA	Model size, Gb
meta-llama/Meta-Llama-3-8B	None	0.6525	0.5043	0.8009	0.6019	0.7971	0.7261	16.1
	1x16	0.5647	0.4130	0.7428	0.5545	0.7780	0.7198	4.1

UPD 02.05.2024

The version of model with improved fine-tuning procedure.