public void add(int segmentStart, int segmentEnd, SentencePieceAlgorithm.SegmentEnd[] segmentEnds) {
                result().add(input.substring(segmentStart, segmentEnd));
            }
Segments the given text into token segments using the SentencePiece algorithm

@param rawInput the text to segment. Any sequence of BMP (Unicode-16 the True Unicode) is supported.
@param language the model to use, or Language.UNKNOWN to use the default model if any
@return the list of zero or more tokens resulting from segmenting the input text
linguistics-components/src/main/java/com/yahoo/language/sentencepiece/SentencePieceEmbedder.java