Guys, Correct me if I am wrong, but to my knowledge peer class loading doesn't *really* work for CQs regardless of autoUnsubscribe flag. The issue is that filter is distributed via discovery messages when new node joins, but discovery uses JDK marshaller that is not aware of p2p loading. So classes are remotely loaded only on initial query execution, but not for the new nodes that join after it's already deployed.
If that's the case, we should just find a right way of dynamic class loading that always works. Deployment SPI seems to be a good candidate, just like for services. -Val On Mon, Oct 22, 2018 at 1:46 AM Denis Mekhanikov <dmekhani...@gmail.com> wrote: > Ilya, > > Disabling P2P class loading for CQ with *autoUnsubscribe=false* is a > minimum, that should be done. > But it will only solves one of the existing problems. > It's still unclear, how to undeploy such queries. > Continuous queries with *autoUnsubscribe=true* have different semantic with > the ones with *autoUnsubscribe=false*. > > The first of them is designed to deliver cache events to the subscriber. > They can't exist without the subscriber node, so they are automatically > undeployed, when the subscriber leaves. > I don't see any problems in using P2P class loading for them. > > The second one is needed to deploy a set of listeners, that are collocated > to data. > In this case the subscriber is not needed, so we don't undeploy the query, > when the subscriber leaves. > But we still have a dependency on the subscriber, since only him can > undeploy the query or redeploy it with different parameters. > Such queries should have a name or Id, that other nodes will use to get a > handle of them. > And P2P class loading is not applicable here, since the query may outlive > the initiator. > > Val, > > We didn't start working on service hot redeployment yet. > We tentatively agreed, that Deployment SPI it a good candidate as a tool > for this task. > > Denis > > чт, 18 окт. 2018 г. в 18:44, Valentin Kulichenko < > valentin.kuliche...@gmail.com>: > > > Denis, > > > > I agree that p2p loading is not the best choice for CQs. Do you know if > > we've already done anything in that area for the Service Grid? Are we > > using/going to use Deployment SPI there as well? Either way, I think it > > would make sense if services and CQs should end up using the same > mechanism > > for dynamic loading. > > > > -Val > > > > On Thu, Oct 18, 2018 at 8:36 AM Ilya Kasnacheev < > ilya.kasnach...@gmail.com > > > > > wrote: > > > > > Hello! > > > > > > Is it possible to simply disable P2P class loading for remote filter of > > > continuous queries with autoUnsubscribe = true? > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > чт, 18 окт. 2018 г. в 18:31, Denis Mekhanikov <dmekhani...@gmail.com>: > > > > > > > Igniters, > > > > > > > > I'm concerned with our continuous query API and deployment procedure. > > > > > > > > Continuous queries have remote filters, that can be deployed over > peer > > > > class loading mechanism (this functionality is currently broken > though: > > > > IGNITE-3653 <https://issues.apache.org/jira/browse/IGNITE-3653>, > > > > IGNITE-9181 > > > > <https://issues.apache.org/jira/browse/IGNITE-9181>). > > > > In usual cases P2P class loading makes sense, but problems begin > when a > > > > node, that deployed the CQ, leaves cluster. By default the CQ is > > > undeployed > > > > in this case. But ContinuousQuery has *autoUnsubscribe* > > > > < > > > > > > > > > > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/ContinuousQuery.html#setAutoUnsubscribe-boolean- > > > > > > > > > property. If set to true, it lets a continuous query stay in cluster, > > > when > > > > initiator leaves. > > > > So, you may end up in a situation, when a continuous query remote > > filter > > > is > > > > deployed in a cluster, but the node, that the class was loaded from, > is > > > > already gone. > > > > New nodes won't have where to load the remote filter class from in > this > > > > case. Also existing nodes may have problems, since class loading of > > > > dependencies of the loaded class happens lazily (and cannot happen > > > > eagerly), so they also need the initiator node. > > > > > > > > The *autoUnsubscribe* property helps to deploy continuous queries, > that > > > > listen to cache updates and react to them in the remote filters. This > > is > > > > pretty convenient, since it allows cache updates be processed > locally, > > > > right where updates happen, without sending data over network. Such > > > > continuous query may not even have a local listener, so initiator > node > > > may > > > > deploy the query and leave. > > > > > > > > *BUT* > > > > > > > > - The deployed CQ becomes impossible to undeploy. It stays in the > > > > cluster till the end of times. > > > > - This case looks like a miss-use of the remote filters. They > should > > > > *filter*, but not *listen* and *react*. Remote filter should be a > > > > stateless predicate rather than a clever callback. > > > > - It causes the problems with P2P class loading, mentioned above. > > > > > > > > So, I'd like to see the *autoUnsubscribe *property deprecated and > > > removed. > > > > > > > > This use-case should be covered by a different mechanism, that will > > allow > > > > to deploy a listener into a cluster by name. It should support > > deployment > > > > SPI <https://apacheignite.readme.io/docs/deployment-spi> instead of > > peer > > > > class loading. It may be built on top of continuous queries, but > have a > > > > more suitable API. > > > > > > > > What do you think? > > > > > > > > Denis > > > > > > > > > >
