How to use EOT_ID
#54
by
saksham-lamini
- opened
Downloading a new model, I see that generation_config.json has "eos_token_id": [128001, 128009], but tokenizer.eos_token_id shows just 128001. How do I update the tokenizer to read the list of values from generation_config?
Thank you!
Also shouldnt tokenizer.eos_token_id also be updated to [128001, 128009] instead of just 128001.
Yes, see here: https://github.com/vllm-project/vllm/issues/4180
Hope this gets fixed by the Huggingface team. I've written a workaround for my application, but this is annoying as well as not flexible as newer models come up.
ArthurZ
changed discussion status to
closed