[
https://issues.apache.org/jira/browse/LUCENE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adrien Grand updated LUCENE-7351:
---------------------------------
Attachment: LUCENE-7351.patch
Updated patch. It now specializes both reading doc ids into an array and
feeding a visitor, which seems to help get the performance back to what it is
on master, or at least less than 1% slower (not easy to distinguish minor
slowdowns to noise at this stage).
It has 3 cases:
- increasing doc ids, which is expected to happen for either sorted segments
or when all docs in a block have the same value. In that case, we delta-encode
using vints.
- doc ids requiring less than 24 bits, which are encoded on 3 bytes.
- doc ids requiring less than 32 bits, which are encoded on 4 bytes like on
master today.
I think it's ready to go?
> BKDWriter should compress doc ids when all values in a block are the same
> -------------------------------------------------------------------------
>
> Key: LUCENE-7351
> URL: https://issues.apache.org/jira/browse/LUCENE-7351
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Priority: Minor
> Attachments: LUCENE-7351.patch, LUCENE-7351.patch
>
>
> BKDWriter writes doc ids using 4 bytes per document. I think it should
> compress similarly to postings when all docs in a block have the same packed
> value. This can happen either when a field has a default value which is
> common across documents or when quantization makes the number of unique
> values so small that a large index will necessarily have blocks that all
> contain the same value (eg. there are only 63490 unique half-float values).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
