@Override
    public List<Integer> embed(String text, Context ctx) {
        var encoding = resolve(ctx.getLanguage()).encode(text);
        return Arrays.stream(encoding.getIds()).mapToInt(Math::toIntExact).boxed().toList();
    }
{@link Embedder}/{@link Segmenter} using Deep Java Library's HuggingFace Tokenizer.

@author bjorncs
linguistics-components/src/main/java/com/yahoo/language/huggingface/HuggingFaceTokenizer.java