inline answers: 2009/12/14 Emmanuel Bernard <emman...@hibernate.org>: > > On 14 déc. 2009, at 11:15, Sanne Grinovero wrote: > >> Even though I'm ok with breaking index compatibility, >> it looks like option "2" will not break compatibility per-se >> (other features might) as they state the same compression algorithm is >> being used, just >> they don't apply it automatically anymore (and the Store.Compress >> isn't defined in 3.0) > > So today (2.4) they use a binary format to store the compressed data?
exactly > >> Actually option "1" (dropping support for Compression) would break it, >> in case you are >> having compressed field. > > right unless we simulate what they did before exactly. > What's the rational for dropping the support on their side? > It was dropped because Lucene was doing behind the scenes something that the user could do himself. Having the user explicitly control the compression strategy enables him to select a specialized compression algorithm best suited for the kind of data. Previously java.util.zip.Deflater was used, which is not always the optimal choice. Also the same field was added multiple times by the IndexWriter (one indexed, one stored compressed), being unexpected by users when not aware of the implementation. >> >> The numbers/dates indexing and new rangequeries will force us to drop >> backwards compatibility, >> unless we implement those later on keeping the old broken strategy. > > You lost me, what is that? You'll find a better explanation in the papers in your dropbox; in short: a date is split in multiple fields, like year-month-day (it's doing better actually, I'm keeping it simple) and so the number of terms is reduced, making a rangequerey not blow up on a tooManyClauses and it also appears to perform better. In this case one field is actually indexed as more fields. The new RangeQuery is smart enough to rewrite himself to a correct boolean query considering the correct fields. As there are many changes already in 3.2 I'm fine moving these improvements further, even if solving the RangeQuery problem is very cool. That's why is was referring to "the old broken strategy": it's not really broken but the new one spares some trouble as you can forget about the tooManyClouses issue (not sure if that is a good thing). WDYT? _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
