Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Mikhail Khludnev
On Fri, Jan 26, 2024 at 11:14 PM Srijan wrote: > Regarding binary field, Solr doesn't allow docvalues for binary field (btw > Lucene does). https://solr.apache.org/guide/solr/latest/indexing-guide/field-types-included-with-solr.html mentions BinaryField FWIW BinaryDocValues has no compression an

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Walter Underwood
You seem to be jumping to conclusions about causes. Might want to step back and do some measurements. Try eliminating parts of the query one at a time, including returning fields. You might need to do this with a query set of a few thousand queries to avoid cache effects. wunder Walter Underwo

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread ufuk yılmaz
Hi, just curious, may I ask how did you come to the conclusion that the compression of fields is the cause of slowness in 9.4? — > On 26 Jan 2024, at 23:13, Srijan wrote: > > I stand corrected. Looks like my stored fields were compressed in Solr 8.11 > too. But something seems to have changed

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Srijan
I stand corrected. Looks like my stored fields were compressed in Solr 8.11 too. But something seems to have changed in 9.x. Decompression is awfully slow. New algorithm? Regarding binary field, Solr doesn't allow docvalues for binary field (btw Lucene does). So I tried using stored binary field b

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Mikhail Khludnev
Hello. Agreed. By default it's BEST_SPEED which is LZ4. So, it can't be faster or less compressive. Binary DocValues Field should be an answer. On Fri, Jan 26, 2024 at 9:41 PM Shawn Heisey wrote: > On 1/26/24 03:38, Srijan wrote: > > Since upgrading to Solr 9.x, I've observed a drastic decrease

Re: Keeping certain stored fields uncompressed

2024-01-26 Thread Shawn Heisey
On 1/26/24 03:38, Srijan wrote: Since upgrading to Solr 9.x, I've observed a drastic decrease in performance – approximately 10 to 20 times slower than before. And this stems from the fact that stored fields in Solr 9.x are now compressed. Decompressing these fields during each search query has i

Keeping certain stored fields uncompressed

2024-01-26 Thread Srijan
Hi All, I'm currently facing a significant performance challenge with Apache Solr 9.x and would greatly appreciate any insights or suggestions you might have. Context: In my Solr setup, I have a custom post filter that is critical to our search process. This filter needs to read a specific stored