private boolean hasOverlappingTokens(PhraseSegmentItem segments) {
        int segmentsLength=0;
        for (Iterator<Item> i = segments.getItemIterator(); i.hasNext(); ) {
            WordItem segment = (WordItem) i.next();
            segmentsLength += segment.getWord().length();
        }
        return segmentsLength > segments.getRawWord().length();
    }
Search to do necessary transforms if the query is in segmented in a CJK language.

@author Steinar Knutsen
container-search/src/main/java/com/yahoo/prelude/querytransform/CJKSearcher.java