Basically, my question is: at which moment services should be deployed on connecting nodes? Should we reject a node from being included into a topology, if services, that are assigned to it, fail to deploy?
It would be good to be sure, that all assigned services are initialised and working, when node start is finished. Otherwise it's unclear, how to notify a user about failures in service initialisation on new nodes. If we decide to provide such guarantee, then how are we going to do that? Is procedure, that I described, viable? It requires hacking through the discovery protocol, which is a thing, that should be avoided. So, maybe there is another way to achieve the same thing? Denis сб, 14 апр. 2018 г. в 1:48, Denis Magda <dma...@apache.org>: > It sounds like it's not a trivial thing to support the automatic services > redeployment after a restart. Let's postpone it for now, guys concentrating > on more urgent things related to the services. > > Alex, Vladimir, > > Could you have a look at Denis question about the discovery-based > deployment? Guess it's the only one thing that prevents us from the IEP > finalization. > > -- > Denis > > On Fri, Apr 13, 2018 at 5:30 AM, Denis Mekhanikov <dmekhani...@gmail.com> > wrote: > > > Vladimir, > > > > Currently we don't save binary metadata to disk, when persistence is > > disabled. > > But we still persist marshaller mappings for some reason, and I > personally > > believe, that we shouldn't. > > > > But I agree, that we should separate data and service persistence > > configuration. > > Right now persistence of services is configured in a pretty non-obvious > > manner. > > It should be a clear way to tell Ignite, whether you want services to be > > persisted or not. > > > > I'm not sure, that we should make "statefullness" in general > configurable. > > Users don't care much, whether metadata is preserved on restarts, or not. > > > > Denis > > > > пт, 13 апр. 2018 г. в 14:29, Vladimir Ozerov <voze...@gridgain.com>: > > > > > Alex, > > > > > > I would say that we've already had this behavior for years - marshaller > > > cache. I think it is time to agree that "in-memory" != stateless. > Instead > > > "in-memory" means "data is not stored on disk". > > > May be we can have a flag which will wipe out all metadata on node > > restart > > > (e.g. it could make sense for embedded clients)? > > > > > > On Fri, Apr 13, 2018 at 12:48 PM, Alexey Goncharuk < > > > alexey.goncha...@gmail.com> wrote: > > > > > > > Denis, > > > > > > > > This is a subtle question. It looks like we have now a number of > > > use-cases > > > > when persistent storage is required even for a pure in-memory mode. > One > > > of > > > > the use-cases is thin client authentication, the other is service > grid > > > > configuration persistence. > > > > > > > > Generally, I would agree that this is an expected behavior. However, > > this > > > > means that a user cannot simply start and stop nodes randomly > anymore. > > > > Ignite start will require some sort of installation or work folder > > > > initialization (sort of initdb in postgres) which is ok for > > > > persistence-enabled modes, but I am not sure if this is expected for > > > > in-memory. Of course, we can run this initialization automatically, > but > > > it > > > > is not always a good idea. > > > > > > > > If we are ok to have this restrictions for in-memory mode, then > service > > > > persistence makes sense. > > > > > > > > --AG > > > > > > > > 2018-04-11 22:36 GMT+03:00 Denis Magda <dma...@apache.org>: > > > > > > > >> Denis, > > > >> > > > >> I think that the service deployment state needs be persisted > > > cluster-wide. > > > >> I guess that our meta-store is capable of doing so. Alex G., > Vladimir, > > > >> could you confirm? > > > >> > > > >> As for the split-brain scenarios, I would put them aside for now > > > because, > > > >> anyway, they have to be solved at lower levels (meta store, > discovery, > > > >> etc.). > > > >> > > > >> Also, I heard that presently we store a service configuration in the > > > >> system > > > >> cache that doesn't give us a way to deploy a new version of a > service > > > >> without undeployment of the previous one. Will this issue be > addressed > > > by > > > >> the new deployment approach? > > > >> > > > >> -- > > > >> Denis > > > >> > > > >> On Wed, Apr 11, 2018 at 1:28 AM, Denis Mekhanikov < > > > dmekhani...@gmail.com> > > > >> wrote: > > > >> > > > >> > Denis, > > > >> > > > > >> > Sounds reasonable. It's not clear, though, what should happen, if > a > > > >> joining > > > >> > node has some services persisted, that are missing on other nodes. > > > >> > Should we deploy them? > > > >> > If we do so, it could lead to surprising behaviour. For example > you > > > >> could > > > >> > kill a node, undeploy a service, then bring back an old node, and > it > > > >> would > > > >> > make the service resurrect. > > > >> > We could store some deployment counter along with the service > > > >> > configurations on all nodes, that would show how many times the > > > service > > > >> > state has changed, i.e. it has been undeployed/redeployed. It > should > > > be > > > >> > kept for undeployed services as well to avoid situations like I > > > >> described. > > > >> > > > > >> > But it still leaves a possibility of incorrect behaviour, if there > > > was a > > > >> > split-brain situation at some point. I don't think we should > precess > > > it > > > >> > somehow, though. If we choose to tackle it, it will overcomplicate > > > >> things > > > >> > for a sake of a minor improvement. > > > >> > > > > >> > Denis > > > >> > > > > >> > вт, 10 апр. 2018 г. в 0:55, Valentin Kulichenko < > > > >> > valentin.kuliche...@gmail.com>: > > > >> > > > > >> > > I was responding to another Denis :) Agree with you on your > point > > > >> though. > > > >> > > > > > >> > > -Val > > > >> > > > > > >> > > On Mon, Apr 9, 2018 at 2:48 PM, Denis Magda <dma...@apache.org> > > > >> wrote: > > > >> > > > > > >> > > > Val, > > > >> > > > > > > >> > > > Guess we're talking about other situations. I'm bringing up > the > > > case > > > >> > > when a > > > >> > > > service was deployed dynamically and has to be brought up > after > > a > > > >> full > > > >> > > > cluster restart w/o user intervention. To achieve this we need > > to > > > >> > persist > > > >> > > > the service's configuration somewhere. > > > >> > > > > > > >> > > > -- > > > >> > > > Denis > > > >> > > > > > > >> > > > On Mon, Apr 9, 2018 at 1:42 PM, Valentin Kulichenko < > > > >> > > > valentin.kuliche...@gmail.com> wrote: > > > >> > > > > > > >> > > > > Denis, > > > >> > > > > > > > >> > > > > EVT_CLASS_DEPLOYED should be fired every time a class is > > > deployed > > > >> or > > > >> > > > > redeployed. If this doesn't happen in some cases, I believe > > this > > > >> > would > > > >> > > > be a > > > >> > > > > bug. I don't think we need to add any new events. > > > >> > > > > > > > >> > > > > -Val > > > >> > > > > > > > >> > > > > On Mon, Apr 9, 2018 at 10:50 AM, Denis Magda < > > dma...@apache.org > > > > > > > >> > > wrote: > > > >> > > > > > > > >> > > > > > Denis, > > > >> > > > > > > > > >> > > > > > I would encourage us to persist a service configuration in > > the > > > >> meta > > > >> > > > store > > > >> > > > > > and have this capability enabled by default. That's > > essential > > > >> for > > > >> > > > > services > > > >> > > > > > started dynamically. Moreover, we support similar behavior > > for > > > >> > > caches, > > > >> > > > > > indexes, and other DDL changes happened at runtime. > > > >> > > > > > > > > >> > > > > > -- > > > >> > > > > > Denis > > > >> > > > > > > > > >> > > > > > On Mon, Apr 9, 2018 at 9:34 AM, Denis Mekhanikov < > > > >> > > > dmekhani...@gmail.com> > > > >> > > > > > wrote: > > > >> > > > > > > > > >> > > > > > > Another question, that I would like to discuss is > whether > > > >> > services > > > >> > > > > should > > > >> > > > > > > be preserved on cluster restarts. > > > >> > > > > > > > > > >> > > > > > > Currently it depends on persistence configuration. If > > > >> persistence > > > >> > > for > > > >> > > > > any > > > >> > > > > > > data region is enabled, then services will be persisted > as > > > >> well. > > > >> > > This > > > >> > > > > is > > > >> > > > > > a > > > >> > > > > > > pretty strange way of configuring this behaviour. > > > >> > > > > > > I'm not sure, if anybody relies on this functionality > > right > > > >> now. > > > >> > > > Should > > > >> > > > > > we > > > >> > > > > > > support it at all? If yes, should we make it > configurable? > > > >> > > > > > > > > > >> > > > > > > Denis > > > >> > > > > > > > > > >> > > > > > > пн, 9 апр. 2018 г. в 19:27, Denis Mekhanikov < > > > >> > > dmekhani...@gmail.com > > > >> > > > >: > > > >> > > > > > > > > > >> > > > > > > > Val, > > > >> > > > > > > > > > > >> > > > > > > > Sounds reasonable. I just think, that user should have > > > some > > > >> way > > > >> > > to > > > >> > > > > > know, > > > >> > > > > > > > that new version of a service class was deployed. > > > >> > > > > > > > One way to do it is to listen to *EVT_CLASS_DEPLOYED. > > *I'm > > > >> not > > > >> > > > sure, > > > >> > > > > > > > whether it is triggered on class redeployment, though. > > If > > > >> not, > > > >> > > then > > > >> > > > > > > another > > > >> > > > > > > > event type should be added. > > > >> > > > > > > > > > > >> > > > > > > > I don't think, that a lot of people will implement > their > > > own > > > >> > > > > > > > *DeploymentSpi*-s, so we should make work with > > > >> > *UriDeploymentSpi* > > > >> > > > as > > > >> > > > > > > > comfortable as possible. > > > >> > > > > > > > > > > >> > > > > > > > Denis > > > >> > > > > > > > > > > >> > > > > > > > пт, 6 апр. 2018 г. в 23:40, Valentin Kulichenko < > > > >> > > > > > > > valentin.kuliche...@gmail.com>: > > > >> > > > > > > > > > > >> > > > > > > >> Yes, the class deployment itself has to be explicit. > > > I.e., > > > >> > there > > > >> > > > has > > > >> > > > > > to > > > >> > > > > > > be > > > >> > > > > > > >> a manual step where user updates the class, and the > > exact > > > >> step > > > >> > > > > > required > > > >> > > > > > > >> would depend on DeploymentSpi implementation. But > then > > > >> Ignite > > > >> > > > takes > > > >> > > > > > care > > > >> > > > > > > >> of > > > >> > > > > > > >> everything else - service redeployment and restart is > > > >> > automatic. > > > >> > > > > > > >> > > > >> > > > > > > >> Dmitriy Pavlov, all this is going to be disabled if > > > >> > > DeploymentSpi > > > >> > > > is > > > >> > > > > > not > > > >> > > > > > > >> configured. In this case service class definitions > have > > > to > > > >> be > > > >> > > > > deployed > > > >> > > > > > > on > > > >> > > > > > > >> local classpath and can't be updated in runtime. Just > > > like > > > >> it > > > >> > > > works > > > >> > > > > > > right > > > >> > > > > > > >> now. > > > >> > > > > > > >> > > > >> > > > > > > >> -Val > > > >> > > > > > > >> > > > >> > > > > > > >> On Fri, Apr 6, 2018 at 10:20 AM, Dmitriy Setrakyan < > > > >> > > > > > > dsetrak...@apache.org > > > >> > > > > > > >> > > > > >> > > > > > > >> wrote: > > > >> > > > > > > >> > > > >> > > > > > > >> > On Fri, Apr 6, 2018 at 9:13 AM, Dmitry Pavlov < > > > >> > > > > > dpavlov....@gmail.com> > > > >> > > > > > > >> > wrote: > > > >> > > > > > > >> > > > > >> > > > > > > >> > > Hi Igniters, > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > I like automatic redeploy which can be disabled > by > > > >> config > > > >> > if > > > >> > > > > user > > > >> > > > > > > >> wants > > > >> > > > > > > >> > to > > > >> > > > > > > >> > > control this process. What do you think? > > > >> > > > > > > >> > > > > > >> > > > > > > >> > > > > >> > > > > > > >> > I do not think we should have anything automatic > when > > > it > > > >> > comes > > > >> > > > to > > > >> > > > > > > >> > deployment, everything should be explicit. However, > > if > > > we > > > >> > use > > > >> > > > the > > > >> > > > > > > >> > deployment SPI, then a user should be able to do > > "hot" > > > >> > > redeploy, > > > >> > > > > > > where a > > > >> > > > > > > >> > new service will be deployed if the user drops an > > > updated > > > >> > jar. > > > >> > > > > > > >> > > > > >> > > > > > > >> > We should not create anything new here. Ignite > > already > > > >> has a > > > >> > > > > > > deployment > > > >> > > > > > > >> SPI > > > >> > > > > > > >> > and it already works in a certain way. Let's not > > change > > > >> it. > > > >> > > > > > > >> > > > > >> > > > > > > >> > D. > > > >> > > > > > > >> > > > > >> > > > > > > >> > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > >
