Hi all [brief overview] I'm evaluating Apache Ignite framework as a replacement for Hazelcast. One of usages where it's planned to be compared is task/job processing. We have implemented tasks management by ourselves based on Hazelcast but not using their MarReduce framework (such as it was very slow in version 3.5). We developed a proprietary framework, which is similar to Ignite's ComputeTaskSplitAdapter and ComputeJobAdapter (failover, task status, distributed execution, etc) at the end and we would like to simplify it by switching to Apache Ignite. To simplify discussion - I've created a trivial project (attached here), which creates one Task with *N* Jobs in it. Jobs are kind of "dummy" - they do nothing and return nothing. The goal is to check an overhead of task management (distribution, failover, etc) overall.
And here are some results: 1. 1 Node - 1 Task - 5000 Jobs -> ~0.8 sec 2. 2 Nodes - 1 Task - 5000 Jobs -> ~10-11 sec 3. 3 Nodes - 1 Task - 5000 Jobs -> ~15 sec 4. 1 Node - 1 Task - 10000 Jobs -> ~3 sec 5. 2 Nodes - 1 Task - 10000 Jobs -> ~40 sec [question] Are these measurements correct from Ignite point of view? Meaning - I didn't miss nothing in term of configuration and this is the price which should be paid for that amount of Jobs to be executed in distributed way (10 sec. for 5000 empty jobs for example). Or it can be tuned/speed-up in some way? Thank you in advance! P.S. And I realized that for a run with 1 Task and 5000 "dummy" jobs Ignite cluster uses almost 3.5Gb heap space. task-management.zip <http://apache-ignite-users.70518.x6.nabble.com/file/t1316/task-management.zip> -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
