@Override
    public List<String> segment(String input, Language language) {
        return List.of(resolve(language).encode(input).getTokens());
    }
{@link Embedder}/{@link Segmenter} using Deep Java Library's HuggingFace Tokenizer.

@author bjorncs
linguistics-components/src/main/java/com/yahoo/language/huggingface/HuggingFaceTokenizer.java