[
https://issues.apache.org/jira/browse/IGNITE-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616263#comment-15616263
]
Andrew Mashenkov commented on IGNITE-4106:
------------------------------------------
I've implementes 2 prototypes. In both I try to speed up SQL query Map phase
with multi-threading approach.
Compared scenarios: 1 node with splitting into 4 threads vs 4 nodes without
splitting.
1) The first one. MapQuery message processing splits into several threads. Each
thread runs same query over certain cache local partitions. When all threads
fiished - results merged and return to Reducer. This approach shows significant
speedup, but throughput is 10-15% slower than if we just add more nodes to
grid. Code is far from ideal, i believe we can fix this 10-15% slowdown.
2) The second. I try to split queries with sending more Map queries messages
from query initiator node. But subset of primary partitions for target node
were specified in these messages . So, remote nodes process these messages in
parallel. This approach give worse results, throughput is 50% slower than if we
just add more nodes to grid.
> SQL: parallelize sql queries over cache local partitions
> --------------------------------------------------------
>
> Key: IGNITE-4106
> URL: https://issues.apache.org/jira/browse/IGNITE-4106
> Project: Ignite
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.6, 1.7
> Reporter: Andrew Mashenkov
> Assignee: Andrew Mashenkov
> Labels: performance
>
> If we run SQL query on cache partitioned over several cluster nodes, it will
> be split into several queries running in parallel. But really we will have
> one thread per query on each node.
> So, for now, to improve SQL query performance we need to run more Ignite
> instances or split caches manually.
> It seems to be better to split local SQL queries over cache partitions, so we
> would be able to parallelize SQL query on every single node and utilize CPU
> more efficiently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
