private List<FieldPart> tokenizePretokenized() {
        String[] pre = rawContent.split("\u001F+");
        List<FieldPart> tokenized = new ArrayList<>(pre.length);
        for (int i = 0; i < pre.length; i++) {
            tokenized.add(createToken(pre[i], true));
        }
        return tokenized;
    }
Creates a hit field

@param fieldname The field name
@param content The field content
@param cjk true if the content is CJK text
@param xmlProperty true if this should not quote XML syntax
container-search/src/main/java/com/yahoo/prelude/hitfield/HitField.java