llama.cpp release compatible with model
Hello, first of all thanks for taking the time to quantize this model, I want to know which release of llama.cpp i need for run the model. I tried with the lasted but failed to load the model.
llm_load_tensors: ggml ctx size = 0.11 MB
error loading model: create_tensor: tensor 'blk.0.attn_k.weight' has wrong shape; expected 4096, 4096, got 4096, 1024, 1, 1
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'model/Barcenas-Mistral-7b.Q4_K_M.gguf'
main: error: unable to load model
Or in any case how can i run this one.
Hello, first of all thanks for taking the time to quantize this model, I want to know which release of llama.cpp i need for run the model. I tried with the lasted but failed to load the model.
llm_load_tensors: ggml ctx size = 0.11 MB
error loading model: create_tensor: tensor 'blk.0.attn_k.weight' has wrong shape; expected 4096, 4096, got 4096, 1024, 1, 1
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'model/Barcenas-Mistral-7b.Q4_K_M.gguf'
main: error: unable to load modelOr in any case how can i run this one.
Hi, sorry for the late reply,
you can run it with the LM Studio program
I use it like this.