Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

2023-05-16 Thread Andrey Gura
> Creating a compute job for each record would quickly get unmanageable and > broadcast isn’t equivalent as it doesn’t take into account the possibility of > partitions migrating during calculations. Of course creating a compute job for each record is a waste of resources. Even in current really

Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

2023-05-15 Thread Stephen Darlington
Right, it doesn’t need to be the same — exposing partitions in this manner is weird — but I think we need to support bulk operations in some capacity. Creating a compute job for each record would quickly get unmanageable and broadcast isn’t equivalent as it doesn’t take into account the possibil

Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

2023-05-15 Thread Andrey Gura
> In Ignite 2, for bulk operations there’s an affinityRun method that takes a > partition as a parameter. I don’t see an equivalent here. Is one in the plan? Ideally users don't need to know a partition for execution of computation and the right tool for it is colocation. Method with partitions w

Re: [DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

2023-05-15 Thread Stephen Darlington
In Ignite 2, for bulk operations there’s an affinityRun method that takes a partition as a parameter. I don’t see an equivalent here. Is one in the plan? Are other APIs like apply, async methods, map-reduce being considered in later phases? > On 15 May 2023, at 12:52, Andrey Gura wrote: > > H

[DISCUSSION] IEP-100: Compute API: Phase 1 - Simple remote job execution

2023-05-15 Thread Andrey Gura
Hi, Igniters! Please take a look at the first phase of proposal for Apache ignite 3 Compute API: Simple remote job execution [1]. While most issues are already resolved due to simplicity of the proposal any feedback and ideas will be really useful for the next phases. Thanks! [1] https://cwiki.