private int consumeSpecialToken(int start) {
        SpecialTokens.Token token = getSpecialToken(start);
        if (token == null) return start;
        tokens.add(toToken(token, start, source));
        return start + token.token().length();
    }
Resets this tokenizer and create tokens from the given string.

@param string the string to tokenize
@param defaultIndexName the name of the index to use as default
@param indexFacts information about the indexes we will search
@return a read-only list of tokens. This list can only be used by this thread
container-search/src/main/java/com/yahoo/prelude/query/parser/Tokenizer.java