@Override
    public Tensor embed(String text, Context context, TensorType tensorType) {
        if ( ! validTensorType(tensorType)) {
            throw new IllegalArgumentException("Invalid colbert embedder tensor target destination. " +
                                               "Wanted a mixed 2-d mapped-indexed tensor, got " + tensorType);
        }
        if (context.getDestination().startsWith("query")) {
            return embedQuery(text, context, tensorType);
        } else {
            return embedDocument(text, context, tensorType);
        }
    }
A ColBERT embedder implementation that maps text to multiple vectors, one vector per token subword id.
This embedder uses a HuggingFace tokenizer to produce a token sequence that is then input to a transformer model.

See col-bert-embedder.def for configurable parameters.

@author bergum
model-integration/src/main/java/ai/vespa/embedding/ColBertEmbedder.java