Re: Thin client: compute support
Hi Alex The idea is great. But I have some concerns that probably should be taken into account for design: 1. We need to have the ability to stop a task execution, smth like OP_COMPUTE_CANCEL_TASK operation (client to server) 2. What's about task execution timeout? It may help to the cluster survival for buggy tasks 3. Ignite doesn't have roles/authorization functionality for now. But a task is the risky operation for cluster (for security reasons). Could we add for Ignite configuration new options: - Explicit turning on for compute task support for thin protocol (disabled by default) for whole cluster - Explicit turning on for compute task support for a node - The list of task names (classes) allowed to execute by thin client. 4. Support the labeling for task that may help to investigate issues on cluster (the idea from IEP-34 [1]) 1. https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov wrote: > Hello, Igniters! > > I have plans to start implementation of Compute interface for Ignite thin > client and want to discuss features that should be implemented. > > We already have Compute implementation for binary-rest clients > (GridClientCompute), which have the following functionality: > - Filtering cluster nodes (projection) for compute > - Executing task by the name > > I think we can implement this functionality in a thin client as well. > > First of all, we need some operation types to request a list of all > available nodes and probably node attributes (by a list of nodes). Node > attributes will be helpful if we will decide to implement analog of > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the thin > client. Perhaps they can be requested lazily. > > From the protocol point of view there will be two new operations: > > OP_CLUSTER_GET_NODES > Request: empty > Response: long topologyVersion, int minorTopologyVersion, int nodesCount, > for each node set of node fields (UUID nodeId, Object or String > consistentId, long order, etc) > > OP_CLUSTER_GET_NODE_ATTRIBUTES > Request: int nodesCount, for each node: UUID nodeId > Response: int nodesCount, for each node: int attributesCount, for each node > attribute: String name, Object value > > To execute tasks we need something like these methods in the client API: > Object execute(String task, Object arg) > Future executeAsync(String task, Object arg) > Object affinityExecute(String task, String cache, Object key, Object arg) > Future affinityExecuteAsync(String task, String cache, Object key, > Object arg) > > Which can be mapped to protocol operations: > > OP_COMPUTE_EXECUTE_TASK > Request: UUID nodeId, String taskName, Object arg > Response: Object result > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > Request: String cacheName, Object key, String taskName, Object arg > Response: Object result > > The second operation is needed because we sometimes can't calculate and > connect to affinity node on the client-side (affinity awareness can be > disabled, custom affinity function can be used or there can be no > connection between client and affinity node), but we can make best effort > to send request to target node if affinity awareness is enabled. > > Currently, on the server-side requests always processed synchronously and > responses are sent right after request was processed. To execute long tasks > async we should whether change this logic or introduce some kind two-way > communication between client and server (now only one-way requests from > client to server are allowed). > > Two-way communication can also be useful in the future if we will send some > server-side generated events to clients. > > In case of two-way communication there can be new operations introduced: > > OP_COMPUTE_EXECUTE_TASK (from client to server) > Request: UUID nodeId, String taskName, Object arg > Response: long taskId > > OP_COMPUTE_TASK_FINISHED (from server to client) > Request: taskId, Object result > Response: empty > > The same for affinity requests. > > Also, we can implement not only execute task operation, but some other > operations from IgniteCompute (broadcast, run, call), but it will be useful > only for java thin client. And even with java thin client we should whether > implement peer-class-loading for thin clients (this also requires two-way > client-server communication) or put classes with executed closures to the > server locally. > > What do you think about proposed protocol changes? > Do we need two-way requests between client and server? > Do we need support of compute methods other than "execute task"? > What do you think about peer-class-loading for thin clients? > -- Sergey Kozlov GridGain Systems www.gridgain.com
Re: Thin client: compute support
1. I believe that Cluster operations for Thin Client protocol are already in the works by Alexandr Shapkin. Can't find the ticket though. Alexandr, can you please confirm and attach the ticket number? 2. Proposed changes will work only for Java tasks that are already deployed on server nodes. This is mostly useless for other thin clients we have (Python, PHP, .NET, C++). We should think of a way to make this useful for all clients. For example, we may allow sending tasks in some scripting language like Javascript. Thoughts? On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov wrote: > Hi Alex > > The idea is great. But I have some concerns that probably should be taken > into account for design: > >1. We need to have the ability to stop a task execution, smth like >OP_COMPUTE_CANCEL_TASK operation (client to server) >2. What's about task execution timeout? It may help to the cluster >survival for buggy tasks >3. Ignite doesn't have roles/authorization functionality for now. But a >task is the risky operation for cluster (for security reasons). Could we >add for Ignite configuration new options: > - Explicit turning on for compute task support for thin protocol > (disabled by default) for whole cluster > - Explicit turning on for compute task support for a node > - The list of task names (classes) allowed to execute by thin client. >4. Support the labeling for task that may help to investigate issues on >cluster (the idea from IEP-34 [1]) > > 1. > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov > wrote: > > > Hello, Igniters! > > > > I have plans to start implementation of Compute interface for Ignite thin > > client and want to discuss features that should be implemented. > > > > We already have Compute implementation for binary-rest clients > > (GridClientCompute), which have the following functionality: > > - Filtering cluster nodes (projection) for compute > > - Executing task by the name > > > > I think we can implement this functionality in a thin client as well. > > > > First of all, we need some operation types to request a list of all > > available nodes and probably node attributes (by a list of nodes). Node > > attributes will be helpful if we will decide to implement analog of > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > thin > > client. Perhaps they can be requested lazily. > > > > From the protocol point of view there will be two new operations: > > > > OP_CLUSTER_GET_NODES > > Request: empty > > Response: long topologyVersion, int minorTopologyVersion, int nodesCount, > > for each node set of node fields (UUID nodeId, Object or String > > consistentId, long order, etc) > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > Request: int nodesCount, for each node: UUID nodeId > > Response: int nodesCount, for each node: int attributesCount, for each > node > > attribute: String name, Object value > > > > To execute tasks we need something like these methods in the client API: > > Object execute(String task, Object arg) > > Future executeAsync(String task, Object arg) > > Object affinityExecute(String task, String cache, Object key, Object arg) > > Future affinityExecuteAsync(String task, String cache, Object > key, > > Object arg) > > > > Which can be mapped to protocol operations: > > > > OP_COMPUTE_EXECUTE_TASK > > Request: UUID nodeId, String taskName, Object arg > > Response: Object result > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > Request: String cacheName, Object key, String taskName, Object arg > > Response: Object result > > > > The second operation is needed because we sometimes can't calculate and > > connect to affinity node on the client-side (affinity awareness can be > > disabled, custom affinity function can be used or there can be no > > connection between client and affinity node), but we can make best effort > > to send request to target node if affinity awareness is enabled. > > > > Currently, on the server-side requests always processed synchronously and > > responses are sent right after request was processed. To execute long > tasks > > async we should whether change this logic or introduce some kind two-way > > communication between client and server (now only one-way requests from > > client to server are allowed). > > > > Two-way communication can also be useful in the future if we will send > some > > server-side generated events to clients. > > > > In case of two-way communication there can be new operations introduced: > > > > OP_COMPUTE_EXECUTE_TASK (from client to server) > > Request: UUID nodeId, String taskName, Object arg > > Response: long taskId > > > > OP_COMPUTE_TASK_FINISHED (from server to client) > > Request: taskId, Object result > > Response: empty > > > > The same for affinity requests. > > > > Also, we can implement not only execute task operation, but some other > > op
[jira] [Created] (IGNITE-12385) .NET Thin Client: introduce ClusterGroup API
Alexandr Shapkin created IGNITE-12385: - Summary: .NET Thin Client: introduce ClusterGroup API Key: IGNITE-12385 URL: https://issues.apache.org/jira/browse/IGNITE-12385 Project: Ignite Issue Type: New Feature Components: thin client Affects Versions: 2.7.6 Reporter: Alexandr Shapkin Assignee: Alexandr Shapkin Fix For: 2.9 We need to implement IClusterGroup API for thin clients. Let's start with following methods: * IClusterGroup.ForDotNet() * IClusterGroup.ForAttributes() * IClusterGroup.GetNodes() -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Thin client: compute support
Hi Pavel On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn wrote: > 1. I believe that Cluster operations for Thin Client protocol are already > in the works > by Alexandr Shapkin. Can't find the ticket though. > Alexandr, can you please confirm and attach the ticket number? > > 2. Proposed changes will work only for Java tasks that are already deployed > on server nodes. > This is mostly useless for other thin clients we have (Python, PHP, .NET, > C++). > I don't guess so. The task (execution) is a way to implement own layer for the thin client application. > We should think of a way to make this useful for all clients. > For example, we may allow sending tasks in some scripting language like > Javascript. > Thoughts? > The arbitrary code execution from a remote client must be protected from malicious code. I don't know how it could be designed but without that we open the hole to kill cluster. > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov > wrote: > > > Hi Alex > > > > The idea is great. But I have some concerns that probably should be taken > > into account for design: > > > >1. We need to have the ability to stop a task execution, smth like > >OP_COMPUTE_CANCEL_TASK operation (client to server) > >2. What's about task execution timeout? It may help to the cluster > >survival for buggy tasks > >3. Ignite doesn't have roles/authorization functionality for now. But > a > >task is the risky operation for cluster (for security reasons). Could > we > >add for Ignite configuration new options: > > - Explicit turning on for compute task support for thin protocol > > (disabled by default) for whole cluster > > - Explicit turning on for compute task support for a node > > - The list of task names (classes) allowed to execute by thin > client. > >4. Support the labeling for task that may help to investigate issues > on > >cluster (the idea from IEP-34 [1]) > > > > 1. > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov > > wrote: > > > > > Hello, Igniters! > > > > > > I have plans to start implementation of Compute interface for Ignite > thin > > > client and want to discuss features that should be implemented. > > > > > > We already have Compute implementation for binary-rest clients > > > (GridClientCompute), which have the following functionality: > > > - Filtering cluster nodes (projection) for compute > > > - Executing task by the name > > > > > > I think we can implement this functionality in a thin client as well. > > > > > > First of all, we need some operation types to request a list of all > > > available nodes and probably node attributes (by a list of nodes). Node > > > attributes will be helpful if we will decide to implement analog of > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > > thin > > > client. Perhaps they can be requested lazily. > > > > > > From the protocol point of view there will be two new operations: > > > > > > OP_CLUSTER_GET_NODES > > > Request: empty > > > Response: long topologyVersion, int minorTopologyVersion, int > nodesCount, > > > for each node set of node fields (UUID nodeId, Object or String > > > consistentId, long order, etc) > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > Request: int nodesCount, for each node: UUID nodeId > > > Response: int nodesCount, for each node: int attributesCount, for each > > node > > > attribute: String name, Object value > > > > > > To execute tasks we need something like these methods in the client > API: > > > Object execute(String task, Object arg) > > > Future executeAsync(String task, Object arg) > > > Object affinityExecute(String task, String cache, Object key, Object > arg) > > > Future affinityExecuteAsync(String task, String cache, Object > > key, > > > Object arg) > > > > > > Which can be mapped to protocol operations: > > > > > > OP_COMPUTE_EXECUTE_TASK > > > Request: UUID nodeId, String taskName, Object arg > > > Response: Object result > > > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > > Request: String cacheName, Object key, String taskName, Object arg > > > Response: Object result > > > > > > The second operation is needed because we sometimes can't calculate and > > > connect to affinity node on the client-side (affinity awareness can be > > > disabled, custom affinity function can be used or there can be no > > > connection between client and affinity node), but we can make best > effort > > > to send request to target node if affinity awareness is enabled. > > > > > > Currently, on the server-side requests always processed synchronously > and > > > responses are sent right after request was processed. To execute long > > tasks > > > async we should whether change this logic or introduce some kind > two-way > > > communication between client and server (now only one-way requests from > > > client to server are
Re: Thin client: compute support
Good points, Sergey. Maybe you are right, and Java-based compute without peer deployment is a good first step for thin clients. On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov wrote: > Hi Pavel > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn > wrote: > > > 1. I believe that Cluster operations for Thin Client protocol are already > > in the works > > by Alexandr Shapkin. Can't find the ticket though. > > Alexandr, can you please confirm and attach the ticket number? > > > > 2. Proposed changes will work only for Java tasks that are already > deployed > > on server nodes. > > This is mostly useless for other thin clients we have (Python, PHP, .NET, > > C++). > > > > I don't guess so. The task (execution) is a way to implement own layer for > the thin client application. > > > > We should think of a way to make this useful for all clients. > > For example, we may allow sending tasks in some scripting language like > > Javascript. > > Thoughts? > > > > The arbitrary code execution from a remote client must be protected > from malicious code. > I don't know how it could be designed but without that we open the hole to > kill cluster. > > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov > > wrote: > > > > > Hi Alex > > > > > > The idea is great. But I have some concerns that probably should be > taken > > > into account for design: > > > > > >1. We need to have the ability to stop a task execution, smth like > > >OP_COMPUTE_CANCEL_TASK operation (client to server) > > >2. What's about task execution timeout? It may help to the cluster > > >survival for buggy tasks > > >3. Ignite doesn't have roles/authorization functionality for now. > But > > a > > >task is the risky operation for cluster (for security reasons). > Could > > we > > >add for Ignite configuration new options: > > > - Explicit turning on for compute task support for thin protocol > > > (disabled by default) for whole cluster > > > - Explicit turning on for compute task support for a node > > > - The list of task names (classes) allowed to execute by thin > > client. > > >4. Support the labeling for task that may help to investigate issues > > on > > >cluster (the idea from IEP-34 [1]) > > > > > > 1. > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov < > plehanov.a...@gmail.com> > > > wrote: > > > > > > > Hello, Igniters! > > > > > > > > I have plans to start implementation of Compute interface for Ignite > > thin > > > > client and want to discuss features that should be implemented. > > > > > > > > We already have Compute implementation for binary-rest clients > > > > (GridClientCompute), which have the following functionality: > > > > - Filtering cluster nodes (projection) for compute > > > > - Executing task by the name > > > > > > > > I think we can implement this functionality in a thin client as well. > > > > > > > > First of all, we need some operation types to request a list of all > > > > available nodes and probably node attributes (by a list of nodes). > Node > > > > attributes will be helpful if we will decide to implement analog of > > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in > the > > > thin > > > > client. Perhaps they can be requested lazily. > > > > > > > > From the protocol point of view there will be two new operations: > > > > > > > > OP_CLUSTER_GET_NODES > > > > Request: empty > > > > Response: long topologyVersion, int minorTopologyVersion, int > > nodesCount, > > > > for each node set of node fields (UUID nodeId, Object or String > > > > consistentId, long order, etc) > > > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > > Request: int nodesCount, for each node: UUID nodeId > > > > Response: int nodesCount, for each node: int attributesCount, for > each > > > node > > > > attribute: String name, Object value > > > > > > > > To execute tasks we need something like these methods in the client > > API: > > > > Object execute(String task, Object arg) > > > > Future executeAsync(String task, Object arg) > > > > Object affinityExecute(String task, String cache, Object key, Object > > arg) > > > > Future affinityExecuteAsync(String task, String cache, Object > > > key, > > > > Object arg) > > > > > > > > Which can be mapped to protocol operations: > > > > > > > > OP_COMPUTE_EXECUTE_TASK > > > > Request: UUID nodeId, String taskName, Object arg > > > > Response: Object result > > > > > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > > > Request: String cacheName, Object key, String taskName, Object arg > > > > Response: Object result > > > > > > > > The second operation is needed because we sometimes can't calculate > and > > > > connect to affinity node on the client-side (affinity awareness can > be > > > > disabled, custom affinity function can be used or there can be no > > > > connection between client
Re: Thin client: compute support
Hello, Sergey! >> 3. Ignite doesn't have roles/authorization functionality for now. I can't agree with you. We already have authorization functionality in Ignite and for a thin client too [1]. But, compute support for a thin client requires some additional efforts to get an appropriate SecurityContext on a remote node. The list of tasks allowed for subjects, including thin clients, is the area of responsibility of GridSecurityProcessor [2]. 1. org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest 2. org.apache.ignite.internal.processors.security.GridSecurityProcessor чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn : > Good points, Sergey. > Maybe you are right, and Java-based compute without peer deployment is a > good first step for thin clients. > > On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov > wrote: > > > Hi Pavel > > > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn > > wrote: > > > > > 1. I believe that Cluster operations for Thin Client protocol are > already > > > in the works > > > by Alexandr Shapkin. Can't find the ticket though. > > > Alexandr, can you please confirm and attach the ticket number? > > > > > > 2. Proposed changes will work only for Java tasks that are already > > deployed > > > on server nodes. > > > This is mostly useless for other thin clients we have (Python, PHP, > .NET, > > > C++). > > > > > > > I don't guess so. The task (execution) is a way to implement own layer > for > > the thin client application. > > > > > > > We should think of a way to make this useful for all clients. > > > For example, we may allow sending tasks in some scripting language like > > > Javascript. > > > Thoughts? > > > > > > > The arbitrary code execution from a remote client must be protected > > from malicious code. > > I don't know how it could be designed but without that we open the hole > to > > kill cluster. > > > > > > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov > > > wrote: > > > > > > > Hi Alex > > > > > > > > The idea is great. But I have some concerns that probably should be > > taken > > > > into account for design: > > > > > > > >1. We need to have the ability to stop a task execution, smth like > > > >OP_COMPUTE_CANCEL_TASK operation (client to server) > > > >2. What's about task execution timeout? It may help to the cluster > > > >survival for buggy tasks > > > >3. Ignite doesn't have roles/authorization functionality for now. > > But > > > a > > > >task is the risky operation for cluster (for security reasons). > > Could > > > we > > > >add for Ignite configuration new options: > > > > - Explicit turning on for compute task support for thin > protocol > > > > (disabled by default) for whole cluster > > > > - Explicit turning on for compute task support for a node > > > > - The list of task names (classes) allowed to execute by thin > > > client. > > > >4. Support the labeling for task that may help to investigate > issues > > > on > > > >cluster (the idea from IEP-34 [1]) > > > > > > > > 1. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov < > > plehanov.a...@gmail.com> > > > > wrote: > > > > > > > > > Hello, Igniters! > > > > > > > > > > I have plans to start implementation of Compute interface for > Ignite > > > thin > > > > > client and want to discuss features that should be implemented. > > > > > > > > > > We already have Compute implementation for binary-rest clients > > > > > (GridClientCompute), which have the following functionality: > > > > > - Filtering cluster nodes (projection) for compute > > > > > - Executing task by the name > > > > > > > > > > I think we can implement this functionality in a thin client as > well. > > > > > > > > > > First of all, we need some operation types to request a list of all > > > > > available nodes and probably node attributes (by a list of nodes). > > Node > > > > > attributes will be helpful if we will decide to implement analog of > > > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in > > the > > > > thin > > > > > client. Perhaps they can be requested lazily. > > > > > > > > > > From the protocol point of view there will be two new operations: > > > > > > > > > > OP_CLUSTER_GET_NODES > > > > > Request: empty > > > > > Response: long topologyVersion, int minorTopologyVersion, int > > > nodesCount, > > > > > for each node set of node fields (UUID nodeId, Object or String > > > > > consistentId, long order, etc) > > > > > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > > > Request: int nodesCount, for each node: UUID nodeId > > > > > Response: int nodesCount, for each node: int attributesCount, for > > each > > > > node > > > > > attribute: String name, Object value > > > > > > > > > > To execute tasks we need something like these methods in
Re: Thin client: compute support
Guys, thanks for your proposals. Sergey, 1. Sure we can add OP_COMPUTE_CANCEL_TASK, but it's only possible with a two-way requests approach. 2. We can also add a timeout to the protocol, also we can add "flags" field (to pass withNoFailover, withNoResultCache flags). 3. I think having two options (for the whole cluster and for node) is confusing. Which one should be preferred if both set? Cluster usually uses the same configuration for each cluster node, so one option will be enough, the node should check only local flag to decide whether to allow compute task execution or not. About the list of task names allowed to execute - it looks like a temporary workaround. As soon as authorization will work for thin clients this option will be deprecated. Do we really need it? 4. We don't have labeling in IgniteCompute now, we only have withName modificator which is used only if no task name was set for the task class. If we really need labeling we should implement it in Ignite first as an independent ticket and then include it into thin client protocol. Pavel, Implementing some DSL for Ignite is a complicated task and also should be implemented in thick client first as an independent feature. For thin clients, I think we can start with execution of locally deployed on nodes tasks as a first step. чт, 21 нояб. 2019 г. в 11:30, Pavel Tupitsyn : > 1. I believe that Cluster operations for Thin Client protocol are already > in the works > by Alexandr Shapkin. Can't find the ticket though. > Alexandr, can you please confirm and attach the ticket number? > > 2. Proposed changes will work only for Java tasks that are already deployed > on server nodes. > This is mostly useless for other thin clients we have (Python, PHP, .NET, > C++). > We should think of a way to make this useful for all clients. > For example, we may allow sending tasks in some scripting language like > Javascript. > Thoughts? > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov > wrote: > > > Hi Alex > > > > The idea is great. But I have some concerns that probably should be taken > > into account for design: > > > >1. We need to have the ability to stop a task execution, smth like > >OP_COMPUTE_CANCEL_TASK operation (client to server) > >2. What's about task execution timeout? It may help to the cluster > >survival for buggy tasks > >3. Ignite doesn't have roles/authorization functionality for now. But > a > >task is the risky operation for cluster (for security reasons). Could > we > >add for Ignite configuration new options: > > - Explicit turning on for compute task support for thin protocol > > (disabled by default) for whole cluster > > - Explicit turning on for compute task support for a node > > - The list of task names (classes) allowed to execute by thin > client. > >4. Support the labeling for task that may help to investigate issues > on > >cluster (the idea from IEP-34 [1]) > > > > 1. > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov > > wrote: > > > > > Hello, Igniters! > > > > > > I have plans to start implementation of Compute interface for Ignite > thin > > > client and want to discuss features that should be implemented. > > > > > > We already have Compute implementation for binary-rest clients > > > (GridClientCompute), which have the following functionality: > > > - Filtering cluster nodes (projection) for compute > > > - Executing task by the name > > > > > > I think we can implement this functionality in a thin client as well. > > > > > > First of all, we need some operation types to request a list of all > > > available nodes and probably node attributes (by a list of nodes). Node > > > attributes will be helpful if we will decide to implement analog of > > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > > thin > > > client. Perhaps they can be requested lazily. > > > > > > From the protocol point of view there will be two new operations: > > > > > > OP_CLUSTER_GET_NODES > > > Request: empty > > > Response: long topologyVersion, int minorTopologyVersion, int > nodesCount, > > > for each node set of node fields (UUID nodeId, Object or String > > > consistentId, long order, etc) > > > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > > Request: int nodesCount, for each node: UUID nodeId > > > Response: int nodesCount, for each node: int attributesCount, for each > > node > > > attribute: String name, Object value > > > > > > To execute tasks we need something like these methods in the client > API: > > > Object execute(String task, Object arg) > > > Future executeAsync(String task, Object arg) > > > Object affinityExecute(String task, String cache, Object key, Object > arg) > > > Future affinityExecuteAsync(String task, String cache, Object > > key, > > > Object arg) > > > > > > Which can be mapped to protocol operations: > > > > > > O
Re: Thin client: compute support
Folks, I started to implement the ClusterGroup API. There are two tickets at the moment: - .NET thin client: introduce Cluster API [1] - .NET Thin Client: introduce ClusterGroup API [2] The first one introduces the cluster API and is merged to master. The second ticket is in progress and allows clients to query and filter server nodes. [1] - https://issues.apache.org/jira/browse/IGNITE-11709 [2] - https://issues.apache.org/jira/browse/IGNITE-12385 чт, 21 нояб. 2019 г. в 12:48, Denis Garus : > Hello, Sergey! > > > >> 3. Ignite doesn't have roles/authorization functionality for now. > > > I can't agree with you. > > We already have authorization functionality in Ignite and for a thin client > too [1]. > > But, compute support for a thin client requires some additional efforts to > get an appropriate SecurityContext on a remote node. > > The list of tasks allowed for subjects, including thin clients, is the area > of responsibility of GridSecurityProcessor [2]. > > > >1. > > > org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest >2. org.apache.ignite.internal.processors.security.GridSecurityProcessor > > > чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn : > > > Good points, Sergey. > > Maybe you are right, and Java-based compute without peer deployment is a > > good first step for thin clients. > > > > On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov > > wrote: > > > > > Hi Pavel > > > > > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn > > > wrote: > > > > > > > 1. I believe that Cluster operations for Thin Client protocol are > > already > > > > in the works > > > > by Alexandr Shapkin. Can't find the ticket though. > > > > Alexandr, can you please confirm and attach the ticket number? > > > > > > > > 2. Proposed changes will work only for Java tasks that are already > > > deployed > > > > on server nodes. > > > > This is mostly useless for other thin clients we have (Python, PHP, > > .NET, > > > > C++). > > > > > > > > > > I don't guess so. The task (execution) is a way to implement own layer > > for > > > the thin client application. > > > > > > > > > > We should think of a way to make this useful for all clients. > > > > For example, we may allow sending tasks in some scripting language > like > > > > Javascript. > > > > Thoughts? > > > > > > > > > > The arbitrary code execution from a remote client must be protected > > > from malicious code. > > > I don't know how it could be designed but without that we open the hole > > to > > > kill cluster. > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov > > > > > wrote: > > > > > > > > > Hi Alex > > > > > > > > > > The idea is great. But I have some concerns that probably should be > > > taken > > > > > into account for design: > > > > > > > > > >1. We need to have the ability to stop a task execution, smth > like > > > > >OP_COMPUTE_CANCEL_TASK operation (client to server) > > > > >2. What's about task execution timeout? It may help to the > cluster > > > > >survival for buggy tasks > > > > >3. Ignite doesn't have roles/authorization functionality for > now. > > > But > > > > a > > > > >task is the risky operation for cluster (for security reasons). > > > Could > > > > we > > > > >add for Ignite configuration new options: > > > > > - Explicit turning on for compute task support for thin > > protocol > > > > > (disabled by default) for whole cluster > > > > > - Explicit turning on for compute task support for a node > > > > > - The list of task names (classes) allowed to execute by thin > > > > client. > > > > >4. Support the labeling for task that may help to investigate > > issues > > > > on > > > > >cluster (the idea from IEP-34 [1]) > > > > > > > > > > 1. > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov < > > > plehanov.a...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hello, Igniters! > > > > > > > > > > > > I have plans to start implementation of Compute interface for > > Ignite > > > > thin > > > > > > client and want to discuss features that should be implemented. > > > > > > > > > > > > We already have Compute implementation for binary-rest clients > > > > > > (GridClientCompute), which have the following functionality: > > > > > > - Filtering cluster nodes (projection) for compute > > > > > > - Executing task by the name > > > > > > > > > > > > I think we can implement this functionality in a thin client as > > well. > > > > > > > > > > > > First of all, we need some operation types to request a list of > all > > > > > > available nodes and probably node attributes (by a list of > nodes). > > > Node > > > > > > attributes will be helpful if we will decide to implement analog > of > > > > > > ClusterGroup#forAttribute or ClusterGroup#for
Re: Joining node validation failure event.
Hi, Andrey. I take part in the development of a custom security plugin for Apache Ignite. There is an information security requirement for which node joining failures due to invalid configuration must be handled by the plugin. This is where my case comes from. Are there any objections to the proposed approach? Regards, Mikhail. On 20.11.2019 19:38, Andrey Gura wrote: Hi, Mikhail! Could you please describe the case for this new event? On Wed, Nov 20, 2019 at 12:45 PM Mikhail Petrov wrote: Hello, Igniters. There is a case which requires to handle joining node validation failure in Ignite components and obtain information of the node that tried to join and the reason for the failure. Now, as I see, there is no way to do it. I propose to implement a new event -- NodeValidationFailedEvent -- and record it in case the validation fails. I have created Tiket [1] and PR [2], which shows an example of implementation. Could anyone take a look at it, please? [1] https://issues.apache.org/jira/browse/IGNITE-12380 [2] https://github.com/apache/ignite/pull/7057
Re: Thin client: compute support
Pavel, The proposed solution won't work for PHP, Python, Node.js, but it will work for C++ and .NET, isn't it? We will just have to deploy C++/.NET code in closter (just as in Java). Best Regards, Igor On Thu, Nov 21, 2019 at 12:59 PM Aleksandr Shapkin wrote: > Folks, > > > > I started to implement the ClusterGroup API. > > > > There are two tickets at the moment: > > - .NET thin client: introduce Cluster API [1] > > - .NET Thin Client: introduce ClusterGroup API [2] > > > > The first one introduces the cluster API and is merged to master. > > The second ticket is in progress and allows clients to query and filter > server nodes. > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-11709 > > [2] - https://issues.apache.org/jira/browse/IGNITE-12385 > > чт, 21 нояб. 2019 г. в 12:48, Denis Garus : > > > Hello, Sergey! > > > > > > >> 3. Ignite doesn't have roles/authorization functionality for now. > > > > > > I can't agree with you. > > > > We already have authorization functionality in Ignite and for a thin > client > > too [1]. > > > > But, compute support for a thin client requires some additional efforts > to > > get an appropriate SecurityContext on a remote node. > > > > The list of tasks allowed for subjects, including thin clients, is the > area > > of responsibility of GridSecurityProcessor [2]. > > > > > > > >1. > > > > > org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest > >2. > org.apache.ignite.internal.processors.security.GridSecurityProcessor > > > > > > чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn : > > > > > Good points, Sergey. > > > Maybe you are right, and Java-based compute without peer deployment is > a > > > good first step for thin clients. > > > > > > On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov > > > wrote: > > > > > > > Hi Pavel > > > > > > > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn < > ptupit...@apache.org> > > > > wrote: > > > > > > > > > 1. I believe that Cluster operations for Thin Client protocol are > > > already > > > > > in the works > > > > > by Alexandr Shapkin. Can't find the ticket though. > > > > > Alexandr, can you please confirm and attach the ticket number? > > > > > > > > > > 2. Proposed changes will work only for Java tasks that are already > > > > deployed > > > > > on server nodes. > > > > > This is mostly useless for other thin clients we have (Python, PHP, > > > .NET, > > > > > C++). > > > > > > > > > > > > > I don't guess so. The task (execution) is a way to implement own > layer > > > for > > > > the thin client application. > > > > > > > > > > > > > We should think of a way to make this useful for all clients. > > > > > For example, we may allow sending tasks in some scripting language > > like > > > > > Javascript. > > > > > Thoughts? > > > > > > > > > > > > > The arbitrary code execution from a remote client must be protected > > > > from malicious code. > > > > I don't know how it could be designed but without that we open the > hole > > > to > > > > kill cluster. > > > > > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov < > skoz...@gridgain.com > > > > > > > > wrote: > > > > > > > > > > > Hi Alex > > > > > > > > > > > > The idea is great. But I have some concerns that probably should > be > > > > taken > > > > > > into account for design: > > > > > > > > > > > >1. We need to have the ability to stop a task execution, smth > > like > > > > > >OP_COMPUTE_CANCEL_TASK operation (client to server) > > > > > >2. What's about task execution timeout? It may help to the > > cluster > > > > > >survival for buggy tasks > > > > > >3. Ignite doesn't have roles/authorization functionality for > > now. > > > > But > > > > > a > > > > > >task is the risky operation for cluster (for security > reasons). > > > > Could > > > > > we > > > > > >add for Ignite configuration new options: > > > > > > - Explicit turning on for compute task support for thin > > > protocol > > > > > > (disabled by default) for whole cluster > > > > > > - Explicit turning on for compute task support for a node > > > > > > - The list of task names (classes) allowed to execute by > thin > > > > > client. > > > > > >4. Support the labeling for task that may help to investigate > > > issues > > > > > on > > > > > >cluster (the idea from IEP-34 [1]) > > > > > > > > > > > > 1. > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 10:58 AM Alex Plehanov < > > > > plehanov.a...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hello, Igniters! > > > > > > > > > > > > > > I have plans to start implementation of Compute interface for > > > Ignite > > > > > thin > > > > > > > client and want to discuss features that should be implemented. > > > > > > > > > > > > > > We already have Compute im
Re: Thin client: compute support
Igor, Yes, we can make it work with some additional support on the server: - Include Platform code along with task name - Invoke Platform when code is not JAVA - Add a way to invoke tasks by name in C++ and .NET On Thu, Nov 21, 2019 at 2:55 PM Igor Sapego wrote: > Pavel, > > The proposed solution won't work for PHP, Python, Node.js, > but it will work for C++ and .NET, isn't it? We will just have > to deploy C++/.NET code in closter (just as in Java). > > Best Regards, > Igor > > > On Thu, Nov 21, 2019 at 12:59 PM Aleksandr Shapkin > wrote: > > > Folks, > > > > > > > > I started to implement the ClusterGroup API. > > > > > > > > There are two tickets at the moment: > > > > - .NET thin client: introduce Cluster API [1] > > > > - .NET Thin Client: introduce ClusterGroup API [2] > > > > > > > > The first one introduces the cluster API and is merged to master. > > > > The second ticket is in progress and allows clients to query and filter > > server nodes. > > > > > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-11709 > > > > [2] - https://issues.apache.org/jira/browse/IGNITE-12385 > > > > чт, 21 нояб. 2019 г. в 12:48, Denis Garus : > > > > > Hello, Sergey! > > > > > > > > > >> 3. Ignite doesn't have roles/authorization functionality for now. > > > > > > > > > I can't agree with you. > > > > > > We already have authorization functionality in Ignite and for a thin > > client > > > too [1]. > > > > > > But, compute support for a thin client requires some additional efforts > > to > > > get an appropriate SecurityContext on a remote node. > > > > > > The list of tasks allowed for subjects, including thin clients, is the > > area > > > of responsibility of GridSecurityProcessor [2]. > > > > > > > > > > > >1. > > > > > > > > > org.apache.ignite.internal.processors.security.client.ThinClientPermissionCheckTest > > >2. > > org.apache.ignite.internal.processors.security.GridSecurityProcessor > > > > > > > > > чт, 21 нояб. 2019 г. в 12:41, Pavel Tupitsyn : > > > > > > > Good points, Sergey. > > > > Maybe you are right, and Java-based compute without peer deployment > is > > a > > > > good first step for thin clients. > > > > > > > > On Thu, Nov 21, 2019 at 12:32 PM Sergey Kozlov > > > > > wrote: > > > > > > > > > Hi Pavel > > > > > > > > > > On Thu, Nov 21, 2019 at 11:30 AM Pavel Tupitsyn < > > ptupit...@apache.org> > > > > > wrote: > > > > > > > > > > > 1. I believe that Cluster operations for Thin Client protocol are > > > > already > > > > > > in the works > > > > > > by Alexandr Shapkin. Can't find the ticket though. > > > > > > Alexandr, can you please confirm and attach the ticket number? > > > > > > > > > > > > 2. Proposed changes will work only for Java tasks that are > already > > > > > deployed > > > > > > on server nodes. > > > > > > This is mostly useless for other thin clients we have (Python, > PHP, > > > > .NET, > > > > > > C++). > > > > > > > > > > > > > > > > I don't guess so. The task (execution) is a way to implement own > > layer > > > > for > > > > > the thin client application. > > > > > > > > > > > > > > > > We should think of a way to make this useful for all clients. > > > > > > For example, we may allow sending tasks in some scripting > language > > > like > > > > > > Javascript. > > > > > > Thoughts? > > > > > > > > > > > > > > > > The arbitrary code execution from a remote client must be protected > > > > > from malicious code. > > > > > I don't know how it could be designed but without that we open the > > hole > > > > to > > > > > kill cluster. > > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 21, 2019 at 11:21 AM Sergey Kozlov < > > skoz...@gridgain.com > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi Alex > > > > > > > > > > > > > > The idea is great. But I have some concerns that probably > should > > be > > > > > taken > > > > > > > into account for design: > > > > > > > > > > > > > >1. We need to have the ability to stop a task execution, > smth > > > like > > > > > > >OP_COMPUTE_CANCEL_TASK operation (client to server) > > > > > > >2. What's about task execution timeout? It may help to the > > > cluster > > > > > > >survival for buggy tasks > > > > > > >3. Ignite doesn't have roles/authorization functionality for > > > now. > > > > > But > > > > > > a > > > > > > >task is the risky operation for cluster (for security > > reasons). > > > > > Could > > > > > > we > > > > > > >add for Ignite configuration new options: > > > > > > > - Explicit turning on for compute task support for thin > > > > protocol > > > > > > > (disabled by default) for whole cluster > > > > > > > - Explicit turning on for compute task support for a node > > > > > > > - The list of task names (classes) allowed to execute by > > thin > > > > > > client. > > > > > > >4. Support the labeling for task that may help to > investigate > > > > issues > > > > > > on > > > > > > >cluster (the idea from IEP-34 [1]) > > > > > > >
[jira] [Created] (IGNITE-12386) The InvalidServerTest.testInvalidServer test is flacky
Amelchev Nikita created IGNITE-12386:
Summary: The InvalidServerTest.testInvalidServer test is flacky
Key: IGNITE-12386
URL: https://issues.apache.org/jira/browse/IGNITE-12386
Project: Ignite
Issue Type: Test
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita
Fix For: 2.8
The InvalidServerTest.testInvalidServer test is flacky.
The reason is that in the test two clusters are started, instead of one.
Because TcpDiscoveryMulticastIpFinder is used by default.
{noformat}
03:41:32] Topology snapshot [ver=1, locNode=10ecdb9c, servers=1, clients=0,
state=INACTIVE, CPUs=5, offheap=19.0GB, heap=2.0GB]
[03:41:36] Topology snapshot [ver=1, locNode=20f59b4f, servers=1, clients=0,
state=INACTIVE, CPUs=5, offheap=19.0GB, heap=2.0GB]
[03:41:40] Topology snapshot [ver=2, locNode=cecf2df6, servers=2, clients=0,
state=INACTIVE, CPUs=5, offheap=19.0GB, heap=2.0GB]
{noformat}
[Fail
example.|https://ci.ignite.apache.org/viewLog.html?tab=buildLog&logTab=tree&filter=debug&expand=all&buildId=4771562&_focus=233870]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12387) .NET Thin Client: Handle unsupported features on older server nodes gracefully
Pavel Tupitsyn created IGNITE-12387: --- Summary: .NET Thin Client: Handle unsupported features on older server nodes gracefully Key: IGNITE-12387 URL: https://issues.apache.org/jira/browse/IGNITE-12387 Project: Ignite Issue Type: Bug Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 2.8 Right now we don't check server version before doing requests for newer features like Affinity Awareness and Cluster API. This causes exceptions like "Invalid re quest op code: 5000", which are cryptic for the user. Fix this: * Affinity Awareness: disable it automatically if there is no server support; log a warning to the log (add logging by adding IgniteClientConfiguration.Logger property) * Individual methods and features like Cluster API - throw an exception with user-friendly explanation -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Thin client: compute support
Alex, what is the primary motivation for this improvement? Are you looking to enable the compute APIs for languages different from Java? - Denis On Wed, Nov 20, 2019 at 11:58 PM Alex Plehanov wrote: > Hello, Igniters! > > I have plans to start implementation of Compute interface for Ignite thin > client and want to discuss features that should be implemented. > > We already have Compute implementation for binary-rest clients > (GridClientCompute), which have the following functionality: > - Filtering cluster nodes (projection) for compute > - Executing task by the name > > I think we can implement this functionality in a thin client as well. > > First of all, we need some operation types to request a list of all > available nodes and probably node attributes (by a list of nodes). Node > attributes will be helpful if we will decide to implement analog of > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the thin > client. Perhaps they can be requested lazily. > > From the protocol point of view there will be two new operations: > > OP_CLUSTER_GET_NODES > Request: empty > Response: long topologyVersion, int minorTopologyVersion, int nodesCount, > for each node set of node fields (UUID nodeId, Object or String > consistentId, long order, etc) > > OP_CLUSTER_GET_NODE_ATTRIBUTES > Request: int nodesCount, for each node: UUID nodeId > Response: int nodesCount, for each node: int attributesCount, for each node > attribute: String name, Object value > > To execute tasks we need something like these methods in the client API: > Object execute(String task, Object arg) > Future executeAsync(String task, Object arg) > Object affinityExecute(String task, String cache, Object key, Object arg) > Future affinityExecuteAsync(String task, String cache, Object key, > Object arg) > > Which can be mapped to protocol operations: > > OP_COMPUTE_EXECUTE_TASK > Request: UUID nodeId, String taskName, Object arg > Response: Object result > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > Request: String cacheName, Object key, String taskName, Object arg > Response: Object result > > The second operation is needed because we sometimes can't calculate and > connect to affinity node on the client-side (affinity awareness can be > disabled, custom affinity function can be used or there can be no > connection between client and affinity node), but we can make best effort > to send request to target node if affinity awareness is enabled. > > Currently, on the server-side requests always processed synchronously and > responses are sent right after request was processed. To execute long tasks > async we should whether change this logic or introduce some kind two-way > communication between client and server (now only one-way requests from > client to server are allowed). > > Two-way communication can also be useful in the future if we will send some > server-side generated events to clients. > > In case of two-way communication there can be new operations introduced: > > OP_COMPUTE_EXECUTE_TASK (from client to server) > Request: UUID nodeId, String taskName, Object arg > Response: long taskId > > OP_COMPUTE_TASK_FINISHED (from server to client) > Request: taskId, Object result > Response: empty > > The same for affinity requests. > > Also, we can implement not only execute task operation, but some other > operations from IgniteCompute (broadcast, run, call), but it will be useful > only for java thin client. And even with java thin client we should whether > implement peer-class-loading for thin clients (this also requires two-way > client-server communication) or put classes with executed closures to the > server locally. > > What do you think about proposed protocol changes? > Do we need two-way requests between client and server? > Do we need support of compute methods other than "execute task"? > What do you think about peer-class-loading for thin clients? >
Re: [RESULT] Ignite PMC Chair Vote
Igniters, The board has officially approved and appointed Dmitry for the role of Ignite PMC Chair. Congrats, Dmitry! Well deserved! Let's move on, there are many things for all of us to do together ;) - Denis On Thu, Nov 7, 2019 at 1:39 PM Dmitriy Pavlov wrote: > Hi Folks, > > Thank you, it would be a great honor for me to serve in this role. And I > really surprised by this community decision (other PMC members have a huge > contribution to the project in different aspects and they > all deserve recognition). > > But it is a little bit early for congrats since the ASF Board appoints new > Chair. It is an advisory decision of project PMC who will be the next > Chair. > > AFAIK next Board meeting is scheduled for November 20. Let's wait for their > decision first. > > Sincerely, > Dmitriy Pavlov > > чт, 7 нояб. 2019 г. в 13:44, Alexey Zinoviev : > > > Dmitry, congrats, we will wait the Community roadmap from you and will > be > > happy to start discussion about that In asf slack or on dev-list! > > > > чт, 7 нояб. 2019 г., 12:27 Petr Ivanov : > > > > > Dmitriy, congratulations! > > > > > > > > > > On 7 Nov 2019, at 12:06, Николай Кулагин > > > wrote: > > > > > > > > Dmitry, my congratulations! > > > > > > > > This is a clear sign that you are moving in the right direction. > > > > Good luck! > > > > > > > > чт, 7 нояб. 2019 г. в 11:12, Вячеслав Коптилин < > > slava.kopti...@gmail.com > > > >: > > > > > > > >> Hello Igniters, Dmitriy, > > > >> > > > >> Congrats, Dmitriy! > > > >> I hope that a board's approval is just a formal step :) > > > >> > > > >> Thanks, > > > >> S. > > > >> > > > >> чт, 7 нояб. 2019 г. в 10:52, Ivan Pavlukhin : > > > >> > > > >>> Great news! > > > >>> > > > >>> Dmitriy my congratulations! I really believe that it is the proper > > > >>> decision today and will make a great positive impact on Ignite > > > >>> Community. > > > >>> > > > >>> Hooray! > > > >>> > > > >>> ср, 6 нояб. 2019 г. в 22:28, Denis Magda : > > > > > > Igniters, > > > > > > As early head-ups for you. Ignite PMCs came to a consensus and > > > selected > > > Dmitry Pavlov as the next Chair. Next, we need to get a board's > > > >> approval. > > > Our request will be reviewed during the next board meeting that > > takes > > > >>> place > > > this month: https://www.apache.org/dev/pmc.html#newchair > > > > > > - > > > Denis > > > > > > > > > On Fri, Nov 1, 2019 at 12:32 PM Denis Magda > > > wrote: > > > > > > > Ignite community, the votes were spread this way. > > > > > > > > Binding (votes of Ignite PMC members): > > > > • Alexey Goncharuk - 3 > > > > • Dmitry Pavlov - 2 > > > > • Nikolay Izhikov - 3 > > > > • Pavel Tupitsyn - 1 > > > > > > > > Non-binding (votes of other community members): > > > > • Alexey Goncharuk - 2 > > > > • Dmitry Pavlov - 11 > > > > • Nikolay Izhikov - 1 > > > > • Pavel Tupitsyn - 0 > > > > > > > > As long as only the binding votes can be counted towards the > > > > election results, we have two candidates who got equal votes - > > Alexey > > > >>> and > > > > Nikolay. Please give Ignite PMC some time to figure out how to > > > >> proceed > > > > further. > > > > > > > > > > > > - > > > > Denis > > > > > > > >>> > > > >>> > > > >>> > > > >>> -- > > > >>> Best regards, > > > >>> Ivan Pavlukhin > > > >>> > > > >> > > > > > > > > >
Re: [RESULT] Ignite PMC Chair Vote
Congrats, Dmitry! Regards, Saikat On Thu, Nov 21, 2019 at 3:05 PM Denis Magda wrote: > Igniters, > > The board has officially approved and appointed Dmitry for the role of > Ignite PMC Chair. > > Congrats, Dmitry! Well deserved! Let's move on, there are many things for > all of us to do together ;) > > - > Denis > > > On Thu, Nov 7, 2019 at 1:39 PM Dmitriy Pavlov wrote: > > > Hi Folks, > > > > Thank you, it would be a great honor for me to serve in this role. And I > > really surprised by this community decision (other PMC members have a > huge > > contribution to the project in different aspects and they > > all deserve recognition). > > > > But it is a little bit early for congrats since the ASF Board appoints > new > > Chair. It is an advisory decision of project PMC who will be the next > > Chair. > > > > AFAIK next Board meeting is scheduled for November 20. Let's wait for > their > > decision first. > > > > Sincerely, > > Dmitriy Pavlov > > > > чт, 7 нояб. 2019 г. в 13:44, Alexey Zinoviev : > > > > > Dmitry, congrats, we will wait the Community roadmap from you and will > > be > > > happy to start discussion about that In asf slack or on dev-list! > > > > > > чт, 7 нояб. 2019 г., 12:27 Petr Ivanov : > > > > > > > Dmitriy, congratulations! > > > > > > > > > > > > > On 7 Nov 2019, at 12:06, Николай Кулагин > > > > wrote: > > > > > > > > > > Dmitry, my congratulations! > > > > > > > > > > This is a clear sign that you are moving in the right direction. > > > > > Good luck! > > > > > > > > > > чт, 7 нояб. 2019 г. в 11:12, Вячеслав Коптилин < > > > slava.kopti...@gmail.com > > > > >: > > > > > > > > > >> Hello Igniters, Dmitriy, > > > > >> > > > > >> Congrats, Dmitriy! > > > > >> I hope that a board's approval is just a formal step :) > > > > >> > > > > >> Thanks, > > > > >> S. > > > > >> > > > > >> чт, 7 нояб. 2019 г. в 10:52, Ivan Pavlukhin >: > > > > >> > > > > >>> Great news! > > > > >>> > > > > >>> Dmitriy my congratulations! I really believe that it is the > proper > > > > >>> decision today and will make a great positive impact on Ignite > > > > >>> Community. > > > > >>> > > > > >>> Hooray! > > > > >>> > > > > >>> ср, 6 нояб. 2019 г. в 22:28, Denis Magda : > > > > > > > > Igniters, > > > > > > > > As early head-ups for you. Ignite PMCs came to a consensus and > > > > selected > > > > Dmitry Pavlov as the next Chair. Next, we need to get a board's > > > > >> approval. > > > > Our request will be reviewed during the next board meeting that > > > takes > > > > >>> place > > > > this month: https://www.apache.org/dev/pmc.html#newchair > > > > > > > > - > > > > Denis > > > > > > > > > > > > On Fri, Nov 1, 2019 at 12:32 PM Denis Magda > > > > wrote: > > > > > > > > > Ignite community, the votes were spread this way. > > > > > > > > > > Binding (votes of Ignite PMC members): > > > > > • Alexey Goncharuk - 3 > > > > > • Dmitry Pavlov - 2 > > > > > • Nikolay Izhikov - 3 > > > > > • Pavel Tupitsyn - 1 > > > > > > > > > > Non-binding (votes of other community members): > > > > > • Alexey Goncharuk - 2 > > > > > • Dmitry Pavlov - 11 > > > > > • Nikolay Izhikov - 1 > > > > > • Pavel Tupitsyn - 0 > > > > > > > > > > As long as only the binding votes can be counted towards the > > > > > election results, we have two candidates who got equal votes - > > > Alexey > > > > >>> and > > > > > Nikolay. Please give Ignite PMC some time to figure out how to > > > > >> proceed > > > > > further. > > > > > > > > > > > > > > > - > > > > > Denis > > > > > > > > > >>> > > > > >>> > > > > >>> > > > > >>> -- > > > > >>> Best regards, > > > > >>> Ivan Pavlukhin > > > > >>> > > > > >> > > > > > > > > > > > > > >
Re: [RESULT] Ignite PMC Chair Vote
Hooray! пт, 22 нояб. 2019 г. в 05:13, Saikat Maitra : > > Congrats, Dmitry! > > Regards, > Saikat > > On Thu, Nov 21, 2019 at 3:05 PM Denis Magda wrote: > > > Igniters, > > > > The board has officially approved and appointed Dmitry for the role of > > Ignite PMC Chair. > > > > Congrats, Dmitry! Well deserved! Let's move on, there are many things for > > all of us to do together ;) > > > > - > > Denis > > > > > > On Thu, Nov 7, 2019 at 1:39 PM Dmitriy Pavlov wrote: > > > > > Hi Folks, > > > > > > Thank you, it would be a great honor for me to serve in this role. And I > > > really surprised by this community decision (other PMC members have a > > huge > > > contribution to the project in different aspects and they > > > all deserve recognition). > > > > > > But it is a little bit early for congrats since the ASF Board appoints > > new > > > Chair. It is an advisory decision of project PMC who will be the next > > > Chair. > > > > > > AFAIK next Board meeting is scheduled for November 20. Let's wait for > > their > > > decision first. > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > чт, 7 нояб. 2019 г. в 13:44, Alexey Zinoviev : > > > > > > > Dmitry, congrats, we will wait the Community roadmap from you and will > > > be > > > > happy to start discussion about that In asf slack or on dev-list! > > > > > > > > чт, 7 нояб. 2019 г., 12:27 Petr Ivanov : > > > > > > > > > Dmitriy, congratulations! > > > > > > > > > > > > > > > > On 7 Nov 2019, at 12:06, Николай Кулагин > > > > > wrote: > > > > > > > > > > > > Dmitry, my congratulations! > > > > > > > > > > > > This is a clear sign that you are moving in the right direction. > > > > > > Good luck! > > > > > > > > > > > > чт, 7 нояб. 2019 г. в 11:12, Вячеслав Коптилин < > > > > slava.kopti...@gmail.com > > > > > >: > > > > > > > > > > > >> Hello Igniters, Dmitriy, > > > > > >> > > > > > >> Congrats, Dmitriy! > > > > > >> I hope that a board's approval is just a formal step :) > > > > > >> > > > > > >> Thanks, > > > > > >> S. > > > > > >> > > > > > >> чт, 7 нояб. 2019 г. в 10:52, Ivan Pavlukhin > >: > > > > > >> > > > > > >>> Great news! > > > > > >>> > > > > > >>> Dmitriy my congratulations! I really believe that it is the > > proper > > > > > >>> decision today and will make a great positive impact on Ignite > > > > > >>> Community. > > > > > >>> > > > > > >>> Hooray! > > > > > >>> > > > > > >>> ср, 6 нояб. 2019 г. в 22:28, Denis Magda : > > > > > > > > > > Igniters, > > > > > > > > > > As early head-ups for you. Ignite PMCs came to a consensus and > > > > > selected > > > > > Dmitry Pavlov as the next Chair. Next, we need to get a board's > > > > > >> approval. > > > > > Our request will be reviewed during the next board meeting that > > > > takes > > > > > >>> place > > > > > this month: https://www.apache.org/dev/pmc.html#newchair > > > > > > > > > > - > > > > > Denis > > > > > > > > > > > > > > > On Fri, Nov 1, 2019 at 12:32 PM Denis Magda > > > > > wrote: > > > > > > > > > > > Ignite community, the votes were spread this way. > > > > > > > > > > > > Binding (votes of Ignite PMC members): > > > > > > • Alexey Goncharuk - 3 > > > > > > • Dmitry Pavlov - 2 > > > > > > • Nikolay Izhikov - 3 > > > > > > • Pavel Tupitsyn - 1 > > > > > > > > > > > > Non-binding (votes of other community members): > > > > > > • Alexey Goncharuk - 2 > > > > > > • Dmitry Pavlov - 11 > > > > > > • Nikolay Izhikov - 1 > > > > > > • Pavel Tupitsyn - 0 > > > > > > > > > > > > As long as only the binding votes can be counted towards the > > > > > > election results, we have two candidates who got equal votes - > > > > Alexey > > > > > >>> and > > > > > > Nikolay. Please give Ignite PMC some time to figure out how to > > > > > >> proceed > > > > > > further. > > > > > > > > > > > > > > > > > > - > > > > > > Denis > > > > > > > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> -- > > > > > >>> Best regards, > > > > > >>> Ivan Pavlukhin > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > -- Best regards, Ivan Pavlukhin
Re: Thin client: compute support
Denis, the primary motivation is to enable execution of deployed to server java tasks from thin clients (java and other languages). пт, 22 нояб. 2019 г. в 00:03, Denis Magda : > Alex, what is the primary motivation for this improvement? Are you looking > to enable the compute APIs for languages different from Java? > > - > Denis > > > On Wed, Nov 20, 2019 at 11:58 PM Alex Plehanov > wrote: > > > Hello, Igniters! > > > > I have plans to start implementation of Compute interface for Ignite thin > > client and want to discuss features that should be implemented. > > > > We already have Compute implementation for binary-rest clients > > (GridClientCompute), which have the following functionality: > > - Filtering cluster nodes (projection) for compute > > - Executing task by the name > > > > I think we can implement this functionality in a thin client as well. > > > > First of all, we need some operation types to request a list of all > > available nodes and probably node attributes (by a list of nodes). Node > > attributes will be helpful if we will decide to implement analog of > > ClusterGroup#forAttribute or ClusterGroup#forePredicate methods in the > thin > > client. Perhaps they can be requested lazily. > > > > From the protocol point of view there will be two new operations: > > > > OP_CLUSTER_GET_NODES > > Request: empty > > Response: long topologyVersion, int minorTopologyVersion, int nodesCount, > > for each node set of node fields (UUID nodeId, Object or String > > consistentId, long order, etc) > > > > OP_CLUSTER_GET_NODE_ATTRIBUTES > > Request: int nodesCount, for each node: UUID nodeId > > Response: int nodesCount, for each node: int attributesCount, for each > node > > attribute: String name, Object value > > > > To execute tasks we need something like these methods in the client API: > > Object execute(String task, Object arg) > > Future executeAsync(String task, Object arg) > > Object affinityExecute(String task, String cache, Object key, Object arg) > > Future affinityExecuteAsync(String task, String cache, Object > key, > > Object arg) > > > > Which can be mapped to protocol operations: > > > > OP_COMPUTE_EXECUTE_TASK > > Request: UUID nodeId, String taskName, Object arg > > Response: Object result > > > > OP_COMPUTE_EXECUTE_TASK_AFFINITY > > Request: String cacheName, Object key, String taskName, Object arg > > Response: Object result > > > > The second operation is needed because we sometimes can't calculate and > > connect to affinity node on the client-side (affinity awareness can be > > disabled, custom affinity function can be used or there can be no > > connection between client and affinity node), but we can make best effort > > to send request to target node if affinity awareness is enabled. > > > > Currently, on the server-side requests always processed synchronously and > > responses are sent right after request was processed. To execute long > tasks > > async we should whether change this logic or introduce some kind two-way > > communication between client and server (now only one-way requests from > > client to server are allowed). > > > > Two-way communication can also be useful in the future if we will send > some > > server-side generated events to clients. > > > > In case of two-way communication there can be new operations introduced: > > > > OP_COMPUTE_EXECUTE_TASK (from client to server) > > Request: UUID nodeId, String taskName, Object arg > > Response: long taskId > > > > OP_COMPUTE_TASK_FINISHED (from server to client) > > Request: taskId, Object result > > Response: empty > > > > The same for affinity requests. > > > > Also, we can implement not only execute task operation, but some other > > operations from IgniteCompute (broadcast, run, call), but it will be > useful > > only for java thin client. And even with java thin client we should > whether > > implement peer-class-loading for thin clients (this also requires two-way > > client-server communication) or put classes with executed closures to the > > server locally. > > > > What do you think about proposed protocol changes? > > Do we need two-way requests between client and server? > > Do we need support of compute methods other than "execute task"? > > What do you think about peer-class-loading for thin clients? > > >
