public Token tokenize(String string, boolean substring) {
        // XXX detonator pattern token.length may be != the length of the
        // matching data in string, ref caseIndependentLength(String)
        String input = toLowerCase(string.substring(0, Math.min(string.length(), maximumLength)));
        for (Iterator<Token> i = tokens.iterator(); i.hasNext();) {
            Token special = i.next();

            if (input.startsWith(special.token())) {
                if (string.length() == special.token().length() || substring || tokenEndsAt(special.token().length(), string))
                    return special;
            }
        }
        return null;
    }
Returns the special token starting at the start of the given string, or null if no
special token starts at this string

@param string the string to search for a special token at the start position
@param substring true to allow the special token to be followed by a character which does not
       mark the end of a token
linguistics/src/main/java/com/yahoo/language/process/SpecialTokens.java