[
https://issues.apache.org/jira/browse/CASSANDRA-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141835#comment-14141835
]
Jay Patel commented on CASSANDRA-7982:
--------------------------------------
Thanks Jonathan for the heads up!
As you can see in the trace output, query on secondary index takes around 300
ms to complete (on a 6 node cluster with RF = 3):
I think because of:
1) Many round trips from coordinator to the same node
2) Multiple indexed scans (one for each token range?) on the same node.
With non-vnode, the same query comes back with 5 to 10 ms, doing only one index
scan per node. I know that even with non-vnode, having lots of nodes will
surely degrade the performance of query on secondary index since it has to go
across most of the nodes (assuming RF of 1 & high cardinality). So, sometime
the best way is to create the custom index. However, we're migrating our
existing code from non-vnode to vnode & hit by the huge increase in latency on
the secondary index queries.
Btw, I still don't totally understand why vnode has to make built-in secondary
index performance this worse for high cardinality/empty table query?
In the case of vnode, why does node need to execute index scans individually
for each token range it holds? In other words, why can't node just do the index
scan for the table data it holds, irrespective of the token ranges held by the
node. If I understood correctly, secondary index is a local index (reverse
look-up table/s) maintained per node basis and do not get distributed across
the cluster so why do token ranges come into picture for secondary index scan?
I may be missing something.
Any clarification will greatly help, or any ideas/thoughts on avoiding multiple
index scans on the same node?
Thanks!
> Batch multiple range requests that are going to the same replica
> -----------------------------------------------------------------
>
> Key: CASSANDRA-7982
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7982
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Jay Patel
> Assignee: Jay Patel
> Fix For: 3.0
>
> Attachments: output1.txt
>
>
> In case of VNode and secondary index query, coordinator sends multiple range
> requests to the same replica. For example, in the attached tracing session
> (output1.txt), coordinator(192.168.51.22) sends multiple requests to
> 192.168.51.25. Why can't we batch all the requests to the same replica to
> avoid multiple round trips? I think this is not the issue with non-vnode
> cluster where each node has one big range (+ replica ranges), instead of many
> small ranges.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
