Hi Mark, With the change, the problem was completely solved!
Sample: (JapaneseAnalyzer) Text: AMeetingWillBeHeldInTheCityHall TokenStream: [A][Meeting][Will][Be][Held][In][The][City][Hall] Query Text: Meeting Output: A<B>Meeting</B>WillBeHeldInTheCityHall Query Text: CityHall Output: AMeetingWillBeHeldInThe<B>City</B><B>Hall</B> Although if I use CJKAnalyzer, which is producing a stream of tokens which all overlap, the problem sill occurs, but with JapaneseAnalyzer, the highlighter works fine. Thank you very much, Koji > -----Original Message----- > From: mark harwood [mailto:[EMAIL PROTECTED] > Sent: Tuesday, September 06, 2005 7:22 PM > To: java-user@lucene.apache.org > Subject: RE: Highlighter apply to Japanese > > > Try change TokenGroup.isDistinct(); > > Maybe the offset test code should be >= rather than > > ie > > boolean isDistinct(Token token) > { > return token.startOffset()>=endOffset; > } > > I've just tried the change with the Junit test and all > seems well still with the non CJK tests. > > > > > ___________________________________________________________ > To help you stay safe and secure online, we've developed the > all new Yahoo! Security Centre. http://uk.security.yahoo.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
