Boyi Dai created SOLR-18559:
-------------------------------
Summary: Updates to a routed alias fail on a node whose cached
aliases are stale
Key: SOLR-18559
URL: https://issues.apache.org/jira/browse/SOLR-18559
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Boyi Dai
RoutedAliasUpdateProcessor.getAliasProps() reads the routed alias properties
only from the node's cached aliases (ZkStateReader.getAliases()). If the cache
doesn't have the alias yet, the update is rejected right away with "Routed
alias <name> appears to have been removed during request processing", even
though the alias exists in ZooKeeper.
A node's cached aliases can be stale when it misses a change to /aliases.json.
For example, AliasesManager.process() catches ConnectionLossException while
re-reading the node after a watch event, logs it, and doesn't re-register the
watch. The cache then stays stale until something else refreshes it. This was
found with fault-injection testing: a ZK connection loss injected while a node
processed the aliases.json watch event made later updates to the routed alias
fail on that node.
This is the same kind of problem as SOLR-12258 and SOLR-12256, which made the
request dispatch path refresh aliases (with sync()) before reporting an unknown
collection or alias. The routed alias update processor doesn't do that.
*Steps to reproduce:*
# Create a routed alias.
# Make one node's cached aliases miss the change (e.g. a ZK connection loss
while it handles the aliases.json watch).
# Send an update for the alias's collection to that node.
# Result: the update fails with "Routed alias ... appears to have been removed
during request processing".
*Expected:* before rejecting the update, refresh the aliases from ZooKeeper
(AliasesManager.update(), which syncs first) and only fail if the alias is
still missing.
*Proposed fix:* do that one refresh in getAliasProps(). I have a patch with a
test and will open a PR.
The watch not being re-registered after a connection loss is a separate issue
(the other watchers in ZkStateReader handle it the same way); this ticket only
covers the update path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
