Also, just for my understanding, is SortedVIntList able to perform some operations such as AND/OR without decompression ?
Some of the algorithms mentioned below claim to do that. But I understand that there are patent issues surrounding these algorithms. http://en.wikipedia.org/wiki/Bitmap_index -Raavan On Sat, Jan 8, 2011 at 10:14 AM, Raavan <valla...@gmail.com> wrote: > Thanks Federico. > > >> my primary concern at the moment is serializing bitsets to recover > searcher warmup time > > I am also considering doing the same to reduce warmup time during restarts. > > It seems one of the disadvantages of SortedVIntList is the performance > skipTo() as per Paul Elschot since it does not support random access like > OpenBitSet. > https://issues.apache.org/jira/browse/LUCENE-1296 > > Our primary concern is memory usage since we have hundreds of filters and > large number of documents. So if the performance is decent, I am thinking of > using SortedVIntList for all our sparse filters. > > -Raavan > > > On Fri, Jan 7, 2011 at 3:11 PM, Federico Fissore <feder...@fissore.org>wrote: > >> First Last, il 07/01/2011 20:55, ha scritto: >> >> Hi, >>> >>> is OpenBitSet / SortedVIntList a compressed bit map index? Which one is >>> better if memory usage is the primary concern ? >>> >>> >> SortedVIntList is compressed, OpenBitSet is not >> >> >> >> Our filters are sparse. So is SortedVIntList better in that case? >>> >>> >> Yes >> >> >> >> Are there any other compressed bitmap index implementations which offer >>> bit >>> map compression at a decent performance assuming filters are sparse? >>> >>> >> I'm too looking for alternative implementations of compressed bitsets, so >> I'm too really interested in everybody experience: my primary concern at the >> moment is serializing bitsets to recover searcher warmup time >> >> I've tried some and roughly tested them: my conclusion was that we (lucene >> users) already stand on the rolls royce of bitset implementations. >> >> federico >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >
