Fix deprecated import for Transformers v5 compatibility

#19

by hmellor HF Staff - opened 11 days ago

←

11 days ago

•

bytes_to_unicode has been available in src/transformers/convert_slow_tokenizer.py for a long time.
bytes_to_unicode will be removed from src/transformers/models/gpt2/tokenization_gpt2.py in Transformers v5.

This PR updates the import, I recommend you do the same for any other custom models you have which import from the old location.

yzhangcs changed pull request status to merged 11 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment