Hunyuen-preview / special_tokens_map.json
codys12's picture
Upload tokenizer
202bc81 verified
raw
history blame contribute delete
187 Bytes
{
"additional_special_tokens": [
"<|startoftext|>",
"<|extra_0|>",
"<|extra_4|>",
"<|extra_5|>",
"<|eos|>"
],
"eos_token": "<|eos|>",
"pad_token": "<|pad|>"
}