Hi Pavel On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn <ptupit...@apache.org> wrote:
> 1. I believe that Cluster operations for Thin Client protocol are already > in the works > by Alexandr Shapkin. Can't find the ticket though. > Alexandr, can you please confirm and attach the ticket number? > > 2. Proposed changes will work only for Java tasks that are already deployed > on server nodes. > This is mostly useless for other thin clients we have (Python, PHP, .NET, > C++). > I don't guess so. The task (execution) is a way to implement own layer for the thin client application. > We should think of a way to make this useful for all clients. > For example, we may allow sending tasks in some scripting language like > Javascript. > Thoughts? > The arbitrary code execution from a remote client must be protected from malicious code. I don't know how it could be designed but without that we open the hole to kill cluster. > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov <skoz...@gridgain.com> > wrote: > > > Hi Alex > > > > The idea is great. But I have some concerns that probably should be taken > > into account for design: > > > > 1. We need to have the ability to stop a task execution, smth like > > OP_COMPUTE_CANCEL_TASK operation (client to server) > > 2. What's about task execution timeout? It may help to the cluster > > survival for buggy tasks > > 3. Ignite doesn't have roles/authorization functionality for now. But > a > > task is the risky operation for cluster (for security reasons). Could > we > > add for Ignite configuration new options: > > - Explicit turning on for compute task support for thin protocol > > (disabled by default) for whole cluster > > - Explicit turning on for compute task support for a node > > - The list of task names (classes) allowed to execute by thin > client. > > 4. Support the labeling for task that may help to investigate issues > on > > cluster (the idea from IEP-34 [1]) > > > > 1. > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov <plehanov.a...@gmail.com> > > wrote: > > > > > Hello, Igniters! > > > > > > I have plans to start implementation of Compute interface for Ignite > thin > > > client and want to discuss features that should be implemented. > > > > > > We already have Compute implementation for binary-rest clients > > > (GridClientCompute), which have the following functionality: > > > - Filtering cluster nodes (projection) for compute > > > - Executing task by the name > > > > > > I think we can implement this functionality in a thin client as well. > > > > > > First of all, we need some operation types to request a list of all > > > available nodes and probably node attributes (by a list of nodes). Node > > > attributes will be helpful if we will decide to implement analog of > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > > thin > > > client. Perhaps they can be requested lazily. > > > > > > From the protocol point of view there will be two new operations: > > > > > > OP_CLUSTER_GET_NODES > > > Request: empty > > > Response: long topologyVersion, int minorTopologyVersion, int > nodesCount, > > > for each node set of node fields (UUID nodeId, Object or String > > > consistentId, long order, etc) > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > Request: int nodesCount, for each node: UUID nodeId > > > Response: int nodesCount, for each node: int attributesCount, for each > > node > > > attribute: String name, Object value > > > > > > To execute tasks we need something like these methods in the client > API: > > > Object execute(String task, Object arg) > > > Future<Object> executeAsync(String task, Object arg) > > > Object affinityExecute(String task, String cache, Object key, Object > arg) > > > Future<Object> affinityExecuteAsync(String task, String cache, Object > > key, > > > Object arg) > > > > > > Which can be mapped to protocol operations: > > > > > > OP_COMPUTE_EXECUTE_TASK > > > Request: UUID nodeId, String taskName, Object arg > > > Response: Object result > > > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > > Request: String cacheName, Object key, String taskName, Object arg > > > Response: Object result > > > > > > The second operation is needed because we sometimes can't calculate and > > > connect to affinity node on the client-side (affinity awareness can be > > > disabled, custom affinity function can be used or there can be no > > > connection between client and affinity node), but we can make best > effort > > > to send request to target node if affinity awareness is enabled. > > > > > > Currently, on the server-side requests always processed synchronously > and > > > responses are sent right after request was processed. To execute long > > tasks > > > async we should whether change this logic or introduce some kind > two-way > > > communication between client and server (now only one-way requests from > > > client to server are allowed). > > > > > > Two-way communication can also be useful in the future if we will send > > some > > > server-side generated events to clients. > > > > > > In case of two-way communication there can be new operations > introduced: > > > > > > OP_COMPUTE_EXECUTE_TASK (from client to server) > > > Request: UUID nodeId, String taskName, Object arg > > > Response: long taskId > > > > > > OP_COMPUTE_TASK_FINISHED (from server to client) > > > Request: taskId, Object result > > > Response: empty > > > > > > The same for affinity requests. > > > > > > Also, we can implement not only execute task operation, but some other > > > operations from IgniteCompute (broadcast, run, call), but it will be > > useful > > > only for java thin client. And even with java thin client we should > > whether > > > implement peer-class-loading for thin clients (this also requires > two-way > > > client-server communication) or put classes with executed closures to > the > > > server locally. > > > > > > What do you think about proposed protocol changes? > > > Do we need two-way requests between client and server? > > > Do we need support of compute methods other than "execute task"? > > > What do you think about peer-class-loading for thin clients? > > > > > > > > > -- > > Sergey Kozlov > > GridGain Systems > > www.gridgain.com > > > -- Sergey Kozlov GridGain Systems www.gridgain.com
