If the schema is no different in the comparison then the problem isn't
SOLR-11078.
You had a question earlier about QTime. QTime ends once Solr identifies
what documents to return, and all the search components have completed
(e.g. faceting, highlighting). After this time, Solr still has to fetch
all the relevant values ("fl" param) out of the index and return them over
the network serialized in whatever format you chose.
On Mon, Dec 9, 2024 at 11:27 AM Tomás Fernández Löbbe <tomasflo...@gmail.com>
wrote:
> Given that this hasn't changed, it's unlikely the degradation problem you
> are seeing, but you are also likely hitting SOLR-11078[1].
>
>
> [1] https://issues.apache.org/jira/browse/SOLR-11078
>
> On Fri, Dec 6, 2024 at 1:08 PM Schmidt, Mihael <mschm...@bauformat.de>
> wrote:
>
>> Hi Tomás,
>>
>>
>>
>> we are using the same schema and the same data in both cases. businessId
>> is a numeric field.
>>
>>
>>
>> Best regards
>>
>>
>>
>> Mihael
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Mihael SchmidtSoftware Engineering Bauformat Küchen GmbH & Co. KG*
>> Kattwinkel 1 | 32584 Löhne | Deutschland
>>
>> Fon: +495732 102-379
>> Fax: +495732 102-300
>> Mail: mschm...@bauformat.de
>> Internet: www.bauformat.de
>>
>>
>>
>> Umsatzsteuer-Identifikationsnummer: *DE 124323068 *- Steuer-Nr.:
>> 310/5705/0461 / Finanzamt Bünde - Handelsregister Bad Oeynhausen HRA 1801
>> Komplementärin: Bauformat Küchen Verwaltungs GmbH - Handelsregister Bad
>> Oeynhausen HRB 1465
>> Geschäftsführer: Michael Assner, Matthias Berens, Sabine Brockschnieder
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Wir erfüllen unsere Informationspflichten zum Datenschutz gem. Artt.
>> 13-14 DS-GVO durch Veröffentlichung auf unserer Internetseite unter:
>> www.bauformat.de/datenschutz oder durch Zusendung auf Ihre formlose
>> Anfrage.
>>
>> *Von:* Tomás Fernández Löbbe <tomasflo...@gmail.com>
>> *Gesendet:* Freitag, 6. Dezember 2024 12:30
>> *An:* users@solr.apache.org
>> *Betreff:* Re: Solr performance much worse in SolrCloud mode ?!
>>
>>
>>
>> Are you using the same schema for both cases? Is “businessId” a numeric
>> field or a String?
>>
>>
>>
>> On Fri, Dec 6, 2024 at 7:53 AM Schmidt, Mihael <mschm...@bauformat.de>
>> wrote:
>>
>> Hi Vincenzo,
>>
>>
>>
>> thanks for the info. There is the query parameters „shards“ which seems
>> to let me choose the shard and replica.
>>
>>
>>
>> Best regards
>>
>>
>>
>> Mihael
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> * Mihael Schmidt Software Engineering Bauformat Küchen GmbH & Co. KG*
>> Kattwinkel 1 | 32584 Löhne | Deutschland
>>
>> Fon: +495732 102-379
>> Fax: +495732 102-300
>> Mail: mschm...@bauformat.de
>> Internet: www.bauformat.de
>>
>>
>>
>> Umsatzsteuer-Identifikationsnummer: *DE 124323068 *- Steuer-Nr.:
>> 310/5705/0461 / Finanzamt Bünde - Handelsregister Bad Oeynhausen HRA 1801
>> Komplementärin: Bauformat Küchen Verwaltungs GmbH - Handelsregister Bad
>> Oeynhausen HRB 1465
>> Geschäftsführer: Michael Assner, Matthias Berens, Sabine Brockschnieder
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Wir erfüllen unsere Informationspflichten zum Datenschutz gem. Artt.
>> 13-14 DS-GVO durch Veröffentlichung auf unserer Internetseite unter:
>> www.bauformat.de/datenschutz oder durch Zusendung auf Ihre formlose
>> Anfrage.
>>
>> *Von:* Vincenzo D'Amore <v.dam...@gmail.com>
>> *Gesendet:* Freitag, 6. Dezember 2024 10:07
>> *An:* users@solr.apache.org
>> *Betreff:* Re: Solr performance much worse in SolrCloud mode ?!
>>
>>
>>
>> I was still investigating it, but have you tried to add a parameter to
>> select the local shard? I remember there was a parameter like
>> preferLocalShards (as far as I remember was deprecated)
>>
>>
>>
>> On Fri, Dec 6, 2024 at 9:51 AM Schmidt, Mihael <mschm...@bauformat.de>
>> wrote:
>>
>> I see QTime values between 3 and 18. And the time it takes till the
>> response is received by the client is around 30 higher. Does that mean that
>> network adds about 30ms to the request?
>>
>>
>>
>> Thanks in advance.
>>
>>
>>
>> Mihael
>>
>>
>>
>>
>>
>>
>>
>> * Mihael Schmidt Software Engineering Bauformat Küchen GmbH & Co. KG*
>> Kattwinkel 1 | 32584 Löhne | Deutschland
>>
>> Fon: +495732 102-379
>> Fax: +495732 102-300
>> Mail: mschm...@bauformat.de
>> Internet: www.bauformat.de
>>
>>
>>
>> Umsatzsteuer-Identifikationsnummer: *DE 124323068 *- Steuer-Nr.:
>> 310/5705/0461 / Finanzamt Bünde - Handelsregister Bad Oeynhausen HRA 1801
>> Komplementärin: Bauformat Küchen Verwaltungs GmbH - Handelsregister Bad
>> Oeynhausen HRB 1465
>> Geschäftsführer: Michael Assner, Matthias Berens, Sabine Brockschnieder
>>
>>
>>
>> Wir erfüllen unsere Informationspflichten zum Datenschutz gem. Artt.
>> 13-14 DS-GVO durch Veröffentlichung auf unserer Internetseite unter:
>> *www.bauformat.de/datenschutz* oder durch Zusendung auf Ihre formlose
>> Anfrage. <https://www.baumannacademy.com/anmelden>
>>
>>
>>
>>
>> *Von: Matthias Krüger <mkrue...@opensourceconnections.com> Gesendet:
>> Mittwoch, 4. Dezember 2024 20:41 An: Schmidt, Mihael
>> <mschm...@bauformat.de> Betreff: Re: Solr performance much worse in
>> SolrCloud mode ?! <https://www.baumannacademy.com/anmelden>*
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Next things I would investigate:
>> <https://www.baumannacademy.com/anmelden>
>>
>> - Is this (at least partially) reproducible outside of this k6 tool,
>> e.g. just sending a few requests using curl?
>> <https://www.baumannacademy.com/anmelden>
>> - What does the response look like, is there a difference in Solr's
>> reported QTime between the standalone vs. SolrCloud setup?
>> <https://www.baumannacademy.com/anmelden>
>>
>> Good luck <https://www.baumannacademy.com/anmelden>
>>
>> Matthias <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> On Tue, Dec 3, 2024 at 2:11 PM Schmidt, Mihael <*mschm...@bauformat.de*>
>> wrote: <https://www.baumannacademy.com/anmelden>
>>
>> Hi Matthias, <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> the tests were done with k6 (*k6.io*) from a server which is on the same
>> LAN as all the targeted servers. I did tests with single and multiple
>> virtual users. The results were all the same. Average response time around
>> 50ms from the Solr cluster setup.
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Request looks like:
>> *https://server/solr-cluster/mdm/select?q=clazz%3Axyz%20AND%20businessId%3A320987*
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> The businessId is randomly chosen from 10000 existing ids. So it is not
>> always the same one queried. <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Each document in the returned JSON has about a dozen attributes and a
>> total of about 400 characters. <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Hope it helps in analyzing it. <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Thanks in advance <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Mihael <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> * Mihael Schmidt Software Engineering Bauformat Küchen GmbH & Co. KG
>> Kattwinkel 1 | 32584 Löhne | Deutschland Fon: +495732 102-379 Fax:
>> +495732 102-300 Mail: mschm...@bauformat.de Internet: www.bauformat.de
>> Umsatzsteuer-Identifikationsnummer: DE 124323068 - Steuer-Nr.:
>> 310/5705/0461 / Finanzamt Bünde - Handelsregister Bad Oeynhausen HRA 1801
>> Komplementärin: Bauformat Küchen Verwaltungs GmbH - Handelsregister Bad
>> Oeynhausen HRB 1465 Geschäftsführer: Michael Assner, Matthias Berens,
>> Sabine Brockschnieder Wir erfüllen unsere Informationspflichten zum
>> Datenschutz gem. Artt. 13-14 DS-GVO durch Veröffentlichung auf unserer
>> Internetseite unter: www.bauformat.de/datenschutz oder durch Zusendung auf
>> Ihre formlose Anfrage. <https://www.baumannacademy.com/anmelden>*
>>
>>
>>
>>
>> *Von: Matthias Krüger <mkrue...@opensourceconnections.com> Gesendet:
>> Dienstag, 3. Dezember 2024 13:12 An: Schmidt, Mihael
>> <mschm...@bauformat.de> Betreff: Re: Solr performance much worse in
>> SolrCloud mode ?! <https://www.baumannacademy.com/anmelden>*
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Hi Mihael, <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> How did you run these tests? What do your queries look like? A p95 of 2ms
>> seems very low. Results probably come completely from cache and the
>> difference might only be in network latency.
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Matthias <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> On Tue, Dec 3, 2024 at 12:51 PM Schmidt, Mihael <*mschm...@bauformat.de*>
>> wrote: <https://www.baumannacademy.com/anmelden>
>>
>> Hi all, <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> We have a standalone Solr instance and want to move to a Solr cluster. We
>> don’t have that many documents in our collection (about 80k) and are very
>> happy with the performance of the standalone Solr instance. We have made
>> some tests with the tool k6 and were surprised.
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Solr standalone (Solr 9.6.1) : <https://www.baumannacademy.com/anmelden>
>>
>> http_req_duration..............: avg=1.81ms min=1ms med=1.6ms
>> max=238.55ms p(90)=2.27ms p(95)=2.69ms
>> <https://www.baumannacademy.com/anmelden>
>>
>> http_reqs......................: 31108 1032.228932/s
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Solr cluster (Solr 9.7.0): <https://www.baumannacademy.com/anmelden>
>>
>> http_req_duration..............: avg=47.98ms min=7.93ms med=48.15ms
>> max=85.51ms p(90)=51.95ms p(95)=53.09ms
>> <https://www.baumannacademy.com/anmelden>
>>
>> http_reqs......................: 1414 41.417403/s
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Both setups have <https://www.baumannacademy.com/anmelden>
>>
>> - a collection with the same documents (about 80k).
>> <https://www.baumannacademy.com/anmelden>
>> - JWT auth configured with the same IDP
>> <https://www.baumannacademy.com/anmelden>
>> - 2GB memory (max heap) <https://www.baumannacademy.com/anmelden>
>> - max file descriptors 1048576
>> <https://www.baumannacademy.com/anmelden>
>> - operating system (Debian Linux 11.5)
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> The Solr cluster has 3 nodes. The used collection is defined with 1 shard
>> and 3 replicas. On the system of each Solr node is also one Zookeeper node.
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Can anybody explain me why the performance is so much worse in SolrCloud
>> mode and how to improve the performance or where to look at?
>> <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Thanks in advance. <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>> Mihael <https://www.baumannacademy.com/anmelden>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> * Mihael Schmidt Software Engineering Bauformat Küchen GmbH & Co. KG
>> Kattwinkel 1 | 32584 Löhne | Deutschland Fon: +495732 102-379 Fax:
>> +495732 102-300 Mail: mschm...@bauformat.de Internet: www.bauformat.de
>> Umsatzsteuer-Identifikationsnummer: DE 124323068 - Steuer-Nr.:
>> 310/5705/0461 / Finanzamt Bünde - Handelsregister Bad Oeynhausen HRA 1801
>> Komplementärin: Bauformat Küchen Verwaltungs GmbH - Handelsregister Bad
>> Oeynhausen HRB 1465 Geschäftsführer: Michael Assner, Matthias Berens,
>> Sabine Brockschnieder Wir erfüllen unsere Informationspflichten zum
>> Datenschutz gem. Artt. 13-14 DS-GVO durch Veröffentlichung auf unserer
>> Internetseite unter: www.bauformat.de/datenschutz oder durch Zusendung auf
>> Ihre formlose Anfrage. <https://www.baumannacademy.com/anmelden>*
>>
>>
>> <https://www.baumannacademy.com/anmelden>
>>
>>
>> -- <https://www.baumannacademy.com/anmelden>
>>
>> Vincenzo D'Amore <https://www.baumannacademy.com/anmelden>
>>
>>
