eezhov-ooma opened a new pull request, #6102:
URL: https://github.com/apache/couchdb/pull/6102
If the local node (coordinator) does not contain the shard,
`group_by_proximity` returns empty list for SameZone shards list.
If the `group_by_proximity` is called with list of shards without local
node, the LocalNode variable is set to 'undefined'. As a result, the SameZone
list is empty and all shards are set to the DifferentZone list.
The solution is to use local node info to define LocalZone var.
----
# Overview
`mem3:group_by_proximity` does not return SameZone nodes if the coordinator
node does not contain the shard.
# Details
The issue found in the CouchDB 3.5.2.
When the request is to get view data like `GET
/{{DB}}/_design/{{ddoc}}/_view/{{view_name}}`, the chain is (the issue ):
```
chttpd_auth_request:authorize_request_int/1 :82 catch-all [_DbName | _]
→ db_authorization_check/1 :108
→ fabric:get_security(DbName, [{user_ctx, Ctx}])
→ fabric.erl :230
→ fabric_util:get_db/2 :127
→ fabric_util:get_shard/4 :145
→ rexi:cast(Node, self(), {fabric_rpc, open_shard, [Name, …]},
[sync])
```
The shard selection here is:
```
fabric_util:get_db(DbName, Options) ->
{Local, SameZone, DifferentZone} =
mem3:group_by_proximity(mem3:shards(DbName)),
Shards = Local ++ lists:keysort(#shard.name, SameZone) ++
lists:keysort(#shard.name, DifferentZone),
Nodes = [node() | nodes()],
Live = [S || #shard{node = N} = S <- Shards, lists:member(N, Nodes)],
...
get_shard(Live, Options, Timeout, Factor).
```
then it sends one request to the first selected shard:
```
fabric_util:get_shard([#shard{node = Node, name = Name} | Rest], Opts,
Timeout, Factor) ->
Mon = rexi_monitor:start([rexi_utils:server_pid(Node)]),
MFA = {fabric_rpc, open_shard, [Name, [{timeout, Timeout} | Opts]]},
Ref = rexi:cast(Node, self(), MFA, [sync]),
```
The issue is with shard selection part. It does not work as expected, and
selects the first node in the alphabetical order from the DifferentZone list
since the SameZone list is empty.
Let's say the DB has the following shards:
```
% curl -s
admin:[email protected]:5984/account%2Fbd%2F6b%2F26370ff1b6aea7e6cfa5ddf27343/_shards
| jq
{
"shards": {
"00000000-ffffffff": [
"[email protected]",
"[email protected]",
"[email protected]"
]
}
}
```
If the request to get view is landed on the `couchdb2-zonea-1.com` then
`mem3:group_by_proximity(mem3:shards(DbName))` returns:
```
([email protected])4>
mem3:group_by_proximity(mem3:shards(<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>)).
{[],[],
[{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]},
{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]},
{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.178729372"...>>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]}]}
```
where `Local = []`, `SameZone = []`, `DifferentZone = [... all shards ...]`.
In opposite, `ushards` uses ZoneMap list to check shards correctly and it
returns SameZone node as expected:
```
([email protected])1>
mem3:ushards(<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>).
[{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]}]
```
The issue is how `group_by_proximity/2` defines the LocalZone variable:
From the example above, the nodes which contain the shards are the
followings. In the code, it is variable "Node".
```
['[email protected]',
'[email protected]',
'[email protected]']
```
Then, we should check what the ZoneMap is. Look the following two code
sections:
```
361 zone_map(Nodes) ->
362 [{Node, node_info(Node, <<"zone">>)} || Node <- Nodes].
```
and
```
117 node_info(Node, Key) ->
118 mem3_nodes:get_node_info(Node, Key).
```
Based on the above code, I ran the following cmd:
```
([email protected])25> [{Node, mem3_nodes:get_node_info(Node,
<<"zone">>)} || Node <- Nodes].
[{'[email protected]',<<"dc3">>},
{'[email protected]',<<"dc1">>},
{'[email protected]',<<"dc2">>}]
```
So, the ZoneMap here should be:
```
[{'[email protected]',<<"dc3">>},
{'[email protected]',<<"dc1">>},
{'[email protected]',<<"dc2">>}]
```
Finally, we need to look at the following code. The line 373 for LocalZone,
it is just wrong. In line 373, the LocalZone is set to undefined. It should be
the zone the local node(coordinator) is in.
```
368 group_by_proximity(Shards, ZoneMap) ->
369 {Local, Remote} = lists:partition(
370 fun(S) -> mem3:node(S) =:= config:node_name() end,
371 Shards
372 ),
373 LocalZone = proplists:get_value(config:node_name(), ZoneMap),
374 Fun = fun(S) -> proplists:get_value(mem3:node(S), ZoneMap) =:=
LocalZone end,
375 {SameZone, DifferentZone} = lists:partition(Fun, Remote),
376 {Local, SameZone, DifferentZone}.
```
The following is an example output I ran from the `couchdb1-zonea-1.com`:
```
([email protected])40>
mem3_nodes:get_node_info(config:node_name(),<<"zone">>).
<<"dc1">>
```
# Solution
The solution is to use `node_info(config:node_name(),<<"zone">>)` instead of
getting value from the ZoneMap, since it does not contain local
node(coordinator).
Example output after patch:
```
([email protected])1>
mem3:group_by_proximity(mem3:shards(<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>)).
{[],
[{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]}],
[{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]},
{shard,<<"shards/00000000-ffffffff/account/bd/6b/26370ff1b6aea7e6cfa5ddf27343.1787293722">>,
'[email protected]',
<<"account/bd/6b/26370ff1b6aea7e6cfa5ddf27343">>,
[0,4294967295],
undefined,[]}]}
([email protected])2>
```
i.e. SameZone contains the shard from the local zone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
