Re: Big number of values for facets

2013-04-27 Thread Michael McCandless
On Fri, Apr 26, 2013 at 12:48 PM, Shai Erera wrote: > You can also try to use a different IntEncoder which compresses the values > better. Try FourFlags and the like. Perhaps it will allow you to index more > facets per document and it will be enough... though i should add "for the > time being" b

Re: Big number of values for facets

2013-04-26 Thread Shai Erera
You can also try to use a different IntEncoder which compresses the values better. Try FourFlags and the like. Perhaps it will allow you to index more facets per document and it will be enough... though i should add "for the time being" b/c according to your scenario, you could easily hit more than

Re: Big number of values for facets

2013-04-26 Thread Nicola Buso
Hi, Mike: no it's not an error of our application I have some entries with this peculiarities :-) probably these cases can be mapped in different ways? If I think to the ER world It's not difficult to have a (n to m) relation between two tables where one of this table is a categorization of some

Re: Big number of values for facets

2013-04-26 Thread Michael McCandless
This means a single document requires more than 32 KB to store all of its ordinals ... so that document must have like at least 6K facets? Are you sure this isn't a bug in your app? That's an insanely high number of facets for one document ... Mike McCandless http://blog.mikemccandless.com On

Re: Big number of values for facets

2013-04-26 Thread Shai Erera
Unfortunately partitions are enabled globally and not per document. And you cannot activate them as you go. It's a setting you need to enable before you index. At least, that's how they currently work - we can think of better ways to do it. Also, partitions were not designed to handle that limitat

Re: Big number of values for facets

2013-04-26 Thread Nicola Buso
Hi Shai, I can't say now how many of these entries I have, I need to trace them, but I expect their are exceptions, like 10 entries no more. Can I enable partitions document by document? Should I activate partitions if I reach a threshold just for these exceptions? Nicola. On Fri, 2013-04-26 a

Re: Big number of values for facets

2013-04-26 Thread Shai Erera
Hi Nicola, I think this limit denotes the number of bytes you can write in a single DV value. So this actually means much less number of facets you index. Do you know how many categories are indexed for that one document? Also, do you expect to index large number of facets for most documents, or