Re: filtering and chaining Collectors

2018-08-16 Thread Michael Sokolov
Right, that makes sense usually. But there are use cases for post-filtering. A good example is when a collector performs grouping or windowing and we want to apply filters based on the grouped or windowed values. On Thu, Aug 16, 2018 at 4:22 AM Adrien Grand wrote: > I think one reason that we do

Re: filtering and chaining Collectors

2018-08-16 Thread Adrien Grand
I think one reason that we don't want to encourage filtering at the collector level is that it is much slower than filtering in the query. The former needs to check hits one by one while the latter can use leap frog to skip documents that don't match. Le mer. 15 août 2018 à 23:27, Michael Sokolov

Re: filtering and chaining Collectors

2018-08-15 Thread Michael Sokolov
Hmm the more I root around, the more crazy it seems to try to thread a return value through all the different places collect() gets called from. Somehow I thought it would just be one place in IndexSearcher somewhere. On Wed, Aug 15, 2018 at 5:18 PM Michael Sokolov wrote: > We have MultiCollecto

filtering and chaining Collectors

2018-08-15 Thread Michael Sokolov
We have MultiCollector to enable running multiple Collectors on the same hits, in sequence for each hit. I think a nice extension would be to enable filtering so that earlier collectors could reject a hit, preventing later collectors from seeing it. This way you could have a post-filter implemente