Val, I'm working on this issue right now. I don't see an ultimate problem here. The discovery data can be deserialized using a system class loader. Serialized representation is stored in *CacheContinuousQueryDeployableObject.* The actual deserialization happens later, when any class loader can be used.
So, I think, peer class loading may be used for continuous queries, but not for the ones with *autoUnsubscribe=false* Denis ср, 24 окт. 2018 г. в 5:35, Valentin Kulichenko < valentin.kuliche...@gmail.com>: > Guys, > > Correct me if I am wrong, but to my knowledge peer class loading doesn't > *really* work for CQs regardless of autoUnsubscribe flag. The issue is that > filter is distributed via discovery messages when new node joins, but > discovery uses JDK marshaller that is not aware of p2p loading. So classes > are remotely loaded only on initial query execution, but not for the new > nodes that join after it's already deployed. > > If that's the case, we should just find a right way of dynamic class > loading that always works. Deployment SPI seems to be a good candidate, > just like for services. > > -Val > > On Mon, Oct 22, 2018 at 1:46 AM Denis Mekhanikov <dmekhani...@gmail.com> > wrote: > > > Ilya, > > > > Disabling P2P class loading for CQ with *autoUnsubscribe=false* is a > > minimum, that should be done. > > But it will only solves one of the existing problems. > > It's still unclear, how to undeploy such queries. > > Continuous queries with *autoUnsubscribe=true* have different semantic > with > > the ones with *autoUnsubscribe=false*. > > > > The first of them is designed to deliver cache events to the subscriber. > > They can't exist without the subscriber node, so they are automatically > > undeployed, when the subscriber leaves. > > I don't see any problems in using P2P class loading for them. > > > > The second one is needed to deploy a set of listeners, that are > collocated > > to data. > > In this case the subscriber is not needed, so we don't undeploy the > query, > > when the subscriber leaves. > > But we still have a dependency on the subscriber, since only him can > > undeploy the query or redeploy it with different parameters. > > Such queries should have a name or Id, that other nodes will use to get a > > handle of them. > > And P2P class loading is not applicable here, since the query may outlive > > the initiator. > > > > Val, > > > > We didn't start working on service hot redeployment yet. > > We tentatively agreed, that Deployment SPI it a good candidate as a tool > > for this task. > > > > Denis > > > > чт, 18 окт. 2018 г. в 18:44, Valentin Kulichenko < > > valentin.kuliche...@gmail.com>: > > > > > Denis, > > > > > > I agree that p2p loading is not the best choice for CQs. Do you know if > > > we've already done anything in that area for the Service Grid? Are we > > > using/going to use Deployment SPI there as well? Either way, I think it > > > would make sense if services and CQs should end up using the same > > mechanism > > > for dynamic loading. > > > > > > -Val > > > > > > On Thu, Oct 18, 2018 at 8:36 AM Ilya Kasnacheev < > > ilya.kasnach...@gmail.com > > > > > > > wrote: > > > > > > > Hello! > > > > > > > > Is it possible to simply disable P2P class loading for remote filter > of > > > > continuous queries with autoUnsubscribe = true? > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > чт, 18 окт. 2018 г. в 18:31, Denis Mekhanikov <dmekhani...@gmail.com > >: > > > > > > > > > Igniters, > > > > > > > > > > I'm concerned with our continuous query API and deployment > procedure. > > > > > > > > > > Continuous queries have remote filters, that can be deployed over > > peer > > > > > class loading mechanism (this functionality is currently broken > > though: > > > > > IGNITE-3653 <https://issues.apache.org/jira/browse/IGNITE-3653>, > > > > > IGNITE-9181 > > > > > <https://issues.apache.org/jira/browse/IGNITE-9181>). > > > > > In usual cases P2P class loading makes sense, but problems begin > > when a > > > > > node, that deployed the CQ, leaves cluster. By default the CQ is > > > > undeployed > > > > > in this case. But ContinuousQuery has *autoUnsubscribe* > > > > > < > > > > > > > > > > > > > > > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/ContinuousQuery.html#setAutoUnsubscribe-boolean- > > > > > > > > > > > property. If set to true, it lets a continuous query stay in > cluster, > > > > when > > > > > initiator leaves. > > > > > So, you may end up in a situation, when a continuous query remote > > > filter > > > > is > > > > > deployed in a cluster, but the node, that the class was loaded > from, > > is > > > > > already gone. > > > > > New nodes won't have where to load the remote filter class from in > > this > > > > > case. Also existing nodes may have problems, since class loading of > > > > > dependencies of the loaded class happens lazily (and cannot happen > > > > > eagerly), so they also need the initiator node. > > > > > > > > > > The *autoUnsubscribe* property helps to deploy continuous queries, > > that > > > > > listen to cache updates and react to them in the remote filters. > This > > > is > > > > > pretty convenient, since it allows cache updates be processed > > locally, > > > > > right where updates happen, without sending data over network. Such > > > > > continuous query may not even have a local listener, so initiator > > node > > > > may > > > > > deploy the query and leave. > > > > > > > > > > *BUT* > > > > > > > > > > - The deployed CQ becomes impossible to undeploy. It stays in > the > > > > > cluster till the end of times. > > > > > - This case looks like a miss-use of the remote filters. They > > should > > > > > *filter*, but not *listen* and *react*. Remote filter should be > a > > > > > stateless predicate rather than a clever callback. > > > > > - It causes the problems with P2P class loading, mentioned > above. > > > > > > > > > > So, I'd like to see the *autoUnsubscribe *property deprecated and > > > > removed. > > > > > > > > > > This use-case should be covered by a different mechanism, that will > > > allow > > > > > to deploy a listener into a cluster by name. It should support > > > deployment > > > > > SPI <https://apacheignite.readme.io/docs/deployment-spi> instead > of > > > peer > > > > > class loading. It may be built on top of continuous queries, but > > have a > > > > > more suitable API. > > > > > > > > > > What do you think? > > > > > > > > > > Denis > > > > > > > > > > > > > > >
