[
https://issues.apache.org/jira/browse/LUCENE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919207#comment-13919207
]
Shai Erera commented on LUCENE-5476:
------------------------------------
Thanks Rob. Few comments:
* Typo in jdoc: "to low" -- "too low"?
* About constructors: can we have two ctors, one taking samplingRatio and
another taking all the parameters?
** And then I think that seed=0 could be used as "generate a seed for me", so
you don't need to take a Long (can take primitive)
* I'd make the first ctor call {{this(false, ratio, 0)}} and fold in init()
inside the all-params ctor.
* It's a matter of personal preference, but I like that static classes are
either at the top of the class (above all class members) or at the end .. but
not in the middle :)
* {{correctCountFacetResults}}
** I think this is redundant in the jdoc "Corrects FacetCounts if sampling is
used." -- you obviously call this method if you used sampling?
* XORShift64Random: maybe we should move it under o.a.l.util (and lucene/core)?
I think it's a useful utility class in general?
** At any rate, it should be static class and let's not nest it inside
SampledDocs
* Now that the collector sub-samples during collection and does not record the
original docs, I think SampledDocs can be declared private (static) class as I
don't see any use for someone to care if it's SampledDocs or just Docs?
bq. Code style-wise: I see this. is sometimes used, and sometimes not. What do
you prefer?
This is a personal preference. I use {{this.}} only when the method has a
parameter which conflicts w/ class members and I often use it explicitly in the
ctor.
bq. It seems FixedBitSet contains a lot of empty space
You can use WAH8DocIdSet which offers better compression, but it means that the
collector would need to declare it doesn't support out-of-order collection. If
{{keepScores=true}} it doesn't support that anyway, but I think you can explore
with it? Can you benchmark with it and report back? If the results are good,
maybe we can turn off out-of-order collection for sampling by default and let
the app override this behavior if it cares about out-of-order?
> Facet sampling
> --------------
>
> Key: LUCENE-5476
> URL: https://issues.apache.org/jira/browse/LUCENE-5476
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Rob Audenaerde
> Attachments: LUCENE-5476.patch, LUCENE-5476.patch, LUCENE-5476.patch,
> SamplingComparison_SamplingFacetsCollector.java, SamplingFacetsCollector.java
>
>
> With LUCENE-5339 facet sampling disappeared.
> When trying to display facet counts on large datasets (>10M documents)
> counting facets is rather expensive, as all the hits are collected and
> processed.
> Sampling greatly reduced this and thus provided a nice speedup. Could it be
> brought back?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
