Hi All,
I want to make sure that no two compute jobs with the same key (any
attribute of Compute job) should be executed in parallel by ignite. I am
using semaphore to achieve this.
Execute method of my compute job:
public Object execute() {
IgniteSemaphore semaphore = this.ignite.semaphore( "123" , 1 , true,
true);
semaphore.aquire();
// run logic
semaphore.release();
}
Using the above, I am able to achieve the goal (no two compute jobs with
semaphore key "123" will be executing in parallel). However, consider the
scenario where I have 1000s of Compute jobs with the same key 123, these
tasks will be distributed across the cluster nodes and all cluster nodes
will start executing these jobs in parallel. But because of the semaphore,
only one job will get executed and all other threads(across the cluster)
will be blocked by the semaphore. This will degrade the performance.
To avoid the blocking, I should not acquire the semaphore inside the job
execution.
But I can not acquire the semaphore on the client-side as that will lead to
synchronous task submission. I want task submission to be asynchronous. What
would be the best place to acquire the semaphore in this scenario?
I am thinking of writing a middle layer using IngiteQueues and
IgniteServices. The client will submit the tasks to IgniteQueue via
IgniteService. IgniteService will try to acquire the semaphore before
submitting the task, if it is not able to acquire that it will push the task
to the end of the queue and pick up the next job. What do you think about
this approach?
Thanks,
Krish
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
