Guys, thanks for your proposals. Sergey, 1. Sure we can add OP_COMPUTE_CANCEL_TASK, but it's only possible with a two-way requests approach. 2. We can also add a timeout to the protocol, also we can add "flags" field (to pass withNoFailover, withNoResultCache flags). 3. I think having two options (for the whole cluster and for node) is confusing. Which one should be preferred if both set? Cluster usually uses the same configuration for each cluster node, so one option will be enough, the node should check only local flag to decide whether to allow compute task execution or not. About the list of task names allowed to execute - it looks like a temporary workaround. As soon as authorization will work for thin clients this option will be deprecated. Do we really need it? 4. We don't have labeling in IgniteCompute now, we only have withName modificator which is used only if no task name was set for the task class. If we really need labeling we should implement it in Ignite first as an independent ticket and then include it into thin client protocol.
Pavel, Implementing some DSL for Ignite is a complicated task and also should be implemented in thick client first as an independent feature. For thin clients, I think we can start with execution of locally deployed on nodes tasks as a first step. чт, 21 нояб. 2019 г. в 11:30, Pavel Tupitsyn <ptupit...@apache.org>: > 1. I believe that Cluster operations for Thin Client protocol are already > in the works > by Alexandr Shapkin. Can't find the ticket though. > Alexandr, can you please confirm and attach the ticket number? > > 2. Proposed changes will work only for Java tasks that are already deployed > on server nodes. > This is mostly useless for other thin clients we have (Python, PHP, .NET, > C++). > We should think of a way to make this useful for all clients. > For example, we may allow sending tasks in some scripting language like > Javascript. > Thoughts? > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov <skoz...@gridgain.com> > wrote: > > > Hi Alex > > > > The idea is great. But I have some concerns that probably should be taken > > into account for design: > > > > 1. We need to have the ability to stop a task execution, smth like > > OP_COMPUTE_CANCEL_TASK operation (client to server) > > 2. What's about task execution timeout? It may help to the cluster > > survival for buggy tasks > > 3. Ignite doesn't have roles/authorization functionality for now. But > a > > task is the risky operation for cluster (for security reasons). Could > we > > add for Ignite configuration new options: > > - Explicit turning on for compute task support for thin protocol > > (disabled by default) for whole cluster > > - Explicit turning on for compute task support for a node > > - The list of task names (classes) allowed to execute by thin > client. > > 4. Support the labeling for task that may help to investigate issues > on > > cluster (the idea from IEP-34 [1]) > > > > 1. > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov <plehanov.a...@gmail.com> > > wrote: > > > > > Hello, Igniters! > > > > > > I have plans to start implementation of Compute interface for Ignite > thin > > > client and want to discuss features that should be implemented. > > > > > > We already have Compute implementation for binary-rest clients > > > (GridClientCompute), which have the following functionality: > > > - Filtering cluster nodes (projection) for compute > > > - Executing task by the name > > > > > > I think we can implement this functionality in a thin client as well. > > > > > > First of all, we need some operation types to request a list of all > > > available nodes and probably node attributes (by a list of nodes). Node > > > attributes will be helpful if we will decide to implement analog of > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > > thin > > > client. Perhaps they can be requested lazily. > > > > > > From the protocol point of view there will be two new operations: > > > > > > OP_CLUSTER_GET_NODES > > > Request: empty > > > Response: long topologyVersion, int minorTopologyVersion, int > nodesCount, > > > for each node set of node fields (UUID nodeId, Object or String > > > consistentId, long order, etc) > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > Request: int nodesCount, for each node: UUID nodeId > > > Response: int nodesCount, for each node: int attributesCount, for each > > node > > > attribute: String name, Object value > > > > > > To execute tasks we need something like these methods in the client > API: > > > Object execute(String task, Object arg) > > > Future<Object> executeAsync(String task, Object arg) > > > Object affinityExecute(String task, String cache, Object key, Object > arg) > > > Future<Object> affinityExecuteAsync(String task, String cache, Object > > key, > > > Object arg) > > > > > > Which can be mapped to protocol operations: > > > > > > OP_COMPUTE_EXECUTE_TASK > > > Request: UUID nodeId, String taskName, Object arg > > > Response: Object result > > > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > > Request: String cacheName, Object key, String taskName, Object arg > > > Response: Object result > > > > > > The second operation is needed because we sometimes can't calculate and > > > connect to affinity node on the client-side (affinity awareness can be > > > disabled, custom affinity function can be used or there can be no > > > connection between client and affinity node), but we can make best > effort > > > to send request to target node if affinity awareness is enabled. > > > > > > Currently, on the server-side requests always processed synchronously > and > > > responses are sent right after request was processed. To execute long > > tasks > > > async we should whether change this logic or introduce some kind > two-way > > > communication between client and server (now only one-way requests from > > > client to server are allowed). > > > > > > Two-way communication can also be useful in the future if we will send > > some > > > server-side generated events to clients. > > > > > > In case of two-way communication there can be new operations > introduced: > > > > > > OP_COMPUTE_EXECUTE_TASK (from client to server) > > > Request: UUID nodeId, String taskName, Object arg > > > Response: long taskId > > > > > > OP_COMPUTE_TASK_FINISHED (from server to client) > > > Request: taskId, Object result > > > Response: empty > > > > > > The same for affinity requests. > > > > > > Also, we can implement not only execute task operation, but some other > > > operations from IgniteCompute (broadcast, run, call), but it will be > > useful > > > only for java thin client. And even with java thin client we should > > whether > > > implement peer-class-loading for thin clients (this also requires > two-way > > > client-server communication) or put classes with executed closures to > the > > > server locally. > > > > > > What do you think about proposed protocol changes? > > > Do we need two-way requests between client and server? > > > Do we need support of compute methods other than "execute task"? > > > What do you think about peer-class-loading for thin clients? > > > > > > > > > -- > > Sergey Kozlov > > GridGain Systems > > www.gridgain.com > > >
