[
https://issues.apache.org/jira/browse/SOLR-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pushkar Raste updated SOLR-9689:
--------------------------------
Attachment: parallelize-peersync.patch
Attached working patch.
For my tests I didn't see much improvement (in fact in some cases performance
degraded) with parallelization. I could not find any hotspot in the profile.
My theory is documents in test are so shorts and simple, that although
parallelizing is working functionally, we need to test this with more complex
documents and verify performance gains.
Most of the parallelization parameters would be subjective and people need to
verify which ones work better for them.
It also seems performance would suffer if there are relatively high DBQs to
applied during DBQs, since updates are applied out of order.
> Process updates concurrently during PeerSync
> --------------------------------------------
>
> Key: SOLR-9689
> URL: https://issues.apache.org/jira/browse/SOLR-9689
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Pushkar Raste
> Attachments: SOLR-9689.patch, SOLR-9689.patch2,
> parallelize-peersync.patch
>
>
> This came up during discussion with [~shalinmangar]
> During {{PeerSync}}, updates are applied one a time by looping through the
> updates received from the leader. This is slow and could keep node in
> recovery for a long time if number of updates to apply were large.
> We can apply updates concurrently, this should be no different than what
> could happen during normal indexing (we can't really ensure that a replica
> will process updates in the same order as the leader or other replicas).
> There are few corner cases around dbq we should be careful about.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
