Error while loading model from path
AttributeError: 'LlamaAttention' object has no attribute 'qkv_proj'
Got the above error while trying to load the model from local. Following is the code used:
model_path="model_llama2_0/"
model_basename="gptq_model-4bit-128g"
tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=True)
model = AutoGPTQForCausalLM.from_quantized(model_path,
model_basename=model_basename,
trust_remote_code=True,
device_map='auto',
use_triton=self.use_triton,
quantize_config=None)
Hmm, I've not heard of that error before. Make sure you've updated Transformers to the latest version, 4.31.0
same error I think the quantize_config lose something to fill in ....but I not know it
Hey. I get the same error. Were you able to figure it out?