On 8/12/2021 12:02 AM, Satya Nand wrote:
Does this alternate format use different data structures to store the
document ids for filters with low document count, Other than the bitmap?
means the size constraint(filter cache size) would apply only on bitmap or
this alternate structure too or their sum?
Yes, the alternate format is just a list of IDs, each of which is a
signed 32-bit integer (four bytes). So it will be far smaller than a
bitmap, as it is only used when the hitcount for the filter is low.
But since there's no way to know in advance whether a particular filter
meets the threshold to use the alternate format, it's best to prepare
for every filterCache entry to be the larger bitmap format. If you take
steps to split your filters into multiple fq parameters to help
performance, chances are that almost all of the cache entries WILL be
the bitmap format.
Thanks,
Shawn
