Hello, Igniters. I think we should support full compute API for the thin clients(at least for the java clients)
I think our roadmap should be the following: 1. Execution existing compute tasks by id (as Alex suggested). 2. Deploy and execution arbitrary compute tasks from the java thin client 3. Research possibility to create API for deployment and execution compute tasks in any language that implemented thin client protocol.(python as a first candidate). > 22 нояб. 2019 г., в 10:19, Alex Plehanov <plehanov.a...@gmail.com> написал(а): > > Denis, the primary motivation is to enable execution of deployed to server > java tasks from thin clients (java and other languages). > > пт, 22 нояб. 2019 г. в 00:03, Denis Magda <dma...@apache.org>: > >> Alex, what is the primary motivation for this improvement? Are you looking >> to enable the compute APIs for languages different from Java? >> >> - >> Denis >> >> >> On Wed, Nov 20, 2019 at 11:58 PM Alex Plehanov <plehanov.a...@gmail.com> >> wrote: >> >>> Hello, Igniters! >>> >>> I have plans to start implementation of Compute interface for Ignite thin >>> client and want to discuss features that should be implemented. >>> >>> We already have Compute implementation for binary-rest clients >>> (GridClientCompute), which have the following functionality: >>> - Filtering cluster nodes (projection) for compute >>> - Executing task by the name >>> >>> I think we can implement this functionality in a thin client as well. >>> >>> First of all, we need some operation types to request a list of all >>> available nodes and probably node attributes (by a list of nodes). Node >>> attributes will be helpful if we will decide to implement analog of >>> ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the >> thin >>> client. Perhaps they can be requested lazily. >>> >>> From the protocol point of view there will be two new operations: >>> >>> OP_CLUSTER_GET_NODES >>> Request: empty >>> Response: long topologyVersion, int minorTopologyVersion, int nodesCount, >>> for each node set of node fields (UUID nodeId, Object or String >>> consistentId, long order, etc) >>> >>> OP_CLUSTER_GET_NODE_ATTRIBUTES >>> Request: int nodesCount, for each node: UUID nodeId >>> Response: int nodesCount, for each node: int attributesCount, for each >> node >>> attribute: String name, Object value >>> >>> To execute tasks we need something like these methods in the client API: >>> Object execute(String task, Object arg) >>> Future<Object> executeAsync(String task, Object arg) >>> Object affinityExecute(String task, String cache, Object key, Object arg) >>> Future<Object> affinityExecuteAsync(String task, String cache, Object >> key, >>> Object arg) >>> >>> Which can be mapped to protocol operations: >>> >>> OP_COMPUTE_EXECUTE_TASK >>> Request: UUID nodeId, String taskName, Object arg >>> Response: Object result >>> >>> OP_COMPUTE_EXECUTE_TASK_AFFINITY >>> Request: String cacheName, Object key, String taskName, Object arg >>> Response: Object result >>> >>> The second operation is needed because we sometimes can't calculate and >>> connect to affinity node on the client-side (affinity awareness can be >>> disabled, custom affinity function can be used or there can be no >>> connection between client and affinity node), but we can make best effort >>> to send request to target node if affinity awareness is enabled. >>> >>> Currently, on the server-side requests always processed synchronously and >>> responses are sent right after request was processed. To execute long >> tasks >>> async we should whether change this logic or introduce some kind two-way >>> communication between client and server (now only one-way requests from >>> client to server are allowed). >>> >>> Two-way communication can also be useful in the future if we will send >> some >>> server-side generated events to clients. >>> >>> In case of two-way communication there can be new operations introduced: >>> >>> OP_COMPUTE_EXECUTE_TASK (from client to server) >>> Request: UUID nodeId, String taskName, Object arg >>> Response: long taskId >>> >>> OP_COMPUTE_TASK_FINISHED (from server to client) >>> Request: taskId, Object result >>> Response: empty >>> >>> The same for affinity requests. >>> >>> Also, we can implement not only execute task operation, but some other >>> operations from IgniteCompute (broadcast, run, call), but it will be >> useful >>> only for java thin client. And even with java thin client we should >> whether >>> implement peer-class-loading for thin clients (this also requires two-way >>> client-server communication) or put classes with executed closures to the >>> server locally. >>> >>> What do you think about proposed protocol changes? >>> Do we need two-way requests between client and server? >>> Do we need support of compute methods other than "execute task"? >>> What do you think about peer-class-loading for thin clients? >>> >>
