Re-Quantize?
#2
by
igoforth
- opened
Hi, thank you for your work.
Would you be willing to update the model to support the latest QuiP# changes? See here https://github.com/Cornell-RelaxML/quip-sharp/issues/31 . According to my understanding, there's no need to recompute the hessians, just to requantize the model.
Oh, quantize more slowly than hessians and https://huggingface.co/KnutJaegersberg/Tess-M-34B-2bit model cannot use, you can use https://github.com/Cornell-RelaxML/quip-sharp/tree/release20231203