Re: highlighter / fragmenter performance for large fields

2008-10-20 Thread Brian Beard
Karsten, Thanks, I will look into this. >Hi Brian, > >I don't know the internals of highlighting („explanation“) in lucene. >But I know that XTF ( >http://xtf.wiki.sourceforge.net/underHood_Documents#tocunderHood_Documents5 >) can handle very large documents (above 100 Mbyte) with highlighting v

Re: highlighter / fragmenter performance for large fields

2008-10-16 Thread Karsten F.
0 milliseconds. I've accomodated the load time for > the text which is about 5-10X faster in general, so 0.1-0.2 seconds for > loading text from the document, and the other 0.8-0.9 performing > highlighting. > -- View this message in context: http://www.nabble.com/highlighter---f

highlighter / fragmenter performance for large fields

2008-10-13 Thread Beard, Brian
We index some documents which have an "all" field containing all of the data which can be searched on. One of the problems we're having is when this field is say 10Mbytes the highlighter takes about a second to calculate the best fragments. The search only takes 30 milliseconds. I've accomodated t