meta-llama

/

Meta-Llama-3-8B-Instruct

like 2.41k

I am trying get rid of "<|start_header_id|>assistant<|end_header_id|>" and "<|eot_id|>" when generating output on TextIteratorStreamer, is there anyway I can achieve this on when generating output?

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True)

model.generate(model_inputs, streamer=streamer, pad_token_id=tokenizer.eos_token_id, max_new_tokens=1000,do_sample=True)