The model repeats the question/answer multiple times in the output
When asking the model a question, I am finding that it either repeats the questions multiple times or the answer multiple times. I am not sure why this is happening. For context, this is my code:
import transformers
import torch
model_id = "meta-llama/Meta-Llama-3-8B"
pipeline = pipeline = transformers.pipeline(
"text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto", max_length=1500)
base_model_answers=[]
for question in user_inputs:
# result = pipe(f"{question}")
result = pipeline(question)
base_model_answers.append(result[0]['generated_text'])
print(result[0]['generated_text'])
This model is a "base" model, it's not supposed to answer questions, but predict the next tokens: more like text generation 😉
Thank you so much, this clarifies things! I am definitely a beginner when it comes to all of this
This model is a "base" model, it's not supposed to answer questions, but predict the next tokens: more like text generation 😉
Which one do you recommend then for llama-3?
Meta-Llama-3-8B-Instruct is the same model but tuned for answering questions.