> To prevent unexpected data loss on deactivation CLI and JMX should require 'force' flag
Vladimir, please, go for it. пт, 7 февр. 2020 г., 11:19 Vladimir Steshin <vlads...@gmail.com>: > Ok. Then we are at the beginning. To prevent unexpected data loss on > deactivation CLI and JMX should require 'force' flag. If there are no extra > proposals I'm going to finish the pull request. > > чт, 6 февр. 2020 г. в 21:43, Alexei Scherbakov < > alexey.scherbak...@gmail.com > >: > > > Vladimir, > > > > IGNITE_REUSE_MEMORY_ON_DEACTIVATE doesn't prevent cache contents from > > clearing. > > It allows allocated memory reuse on re-activation to avoid OS specific > > issues if allocated heap is rather large. > > > > You are right to create separate ticket for implementing required > behavior. > > > > чт, 6 февр. 2020 г. в 16:37, Vladimir Steshin <vlads...@gmail.com>: > > > > > Or, is flag [1] is actual only for persistence mode? Related ticket [2] > > is > > > completely about disabled persistence. > > > If previous assumption is right, I think, we can involve flag [1] in > > ticket > > > [2]. > > > > > > [1] > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614 > > > > > > чт, 6 февр. 2020 г. в 16:10, Vladimir Steshin <vlads...@gmail.com>: > > > > > > > Denis, Alexei, > > > > > > > > Regarding usage of flag > > > > > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE > > > > [1] > > > > > > > > When enabled, I think the following test should work. But it fails. > > > > > > > > //---------------------------------------------------------------- > > > > @Test > > > > public void testDataPresent() throws Exception { > > > > System.setProperty(IGNITE_REUSE_MEMORY_ON_DEACTIVATE, > "true"); > > > > > > > > IgniteEx i = startGrid(0); > > > > > > > > assertFalse( > > > > > > > > > > i.configuration().getDataStorageConfiguration().getDefaultDataRegionConfiguration() > > > > .isPersistenceEnabled() ); > > > > > > > > String name = "non-persistent-cache"; > > > > > > > > i.createCache(name).put(1L, 1L); > > > > > > > > assertEquals(1L, i.cache(name).get(1L)); > > > > > > > > i.cluster().state(ClusterState.INACTIVE); > > > > i.cluster().state(ClusterState.ACTIVE); > > > > > > > > assertEquals(1L, i.cache(name).get(1L)); //Assertion error > > here! > > > > } > > > > //---------------------------------------------------------------- > > > > > > > > > > > > Several notes: > > > > > > > > - IgniteCacheDatabaseSharedManager#reuseMemory > > > > is true > > > > - IgniteCacheDatabaseSharedManager#onDeActivate(boolean shutdown) > > > > is called with shutdown == false > > > > - PageMemoryNoStoreImpl#stop(booleam deallocate) > > > > is called with deallocate == false > > > > > > > > But the cache from the test still has zero size after reactivation. > > > > > > > > Is flag [1] disabled by default because it is not implemented / > doesn't > > > > work? Do we need to skip it in current ticket and rise new one? > > > > > > > > ср, 5 февр. 2020 г. в 21:05, Denis Magda <dma...@apache.org>: > > > > > > > >> I believe there might be a consistency-related reason why this flag > > was > > > >> disabled by default for caches that store data in Ignite native > > > >> persistence. I hope, Alex Goncharuk or Scherbakov can shed some > light > > on > > > >> this. > > > >> > > > >> As for the memory-only caches or caches backed up by a CacheStore > such > > > as > > > >> an RDBMS, enabling of the flag should be harmless. Once we do this, > > > we'll > > > >> eliminate the need to load the data back into the cluster which can > > be a > > > >> time-consuming operation depending on the data volume. > > > >> > > > >> > > > >> - > > > >> Denis > > > >> > > > >> > > > >> On Wed, Feb 5, 2020 at 11:58 AM Vladimir Steshin < > vlads...@gmail.com> > > > >> wrote: > > > >> > > > >> > Denis, but why reuse-data-on-deactivate was disabled by default? > > There > > > >> > should be a reason for that. Any data consistency issues when node > > > gets > > > >> > activated anew? We may use both solutions because the flag can be > > > >> switched > > > >> > off again. > > > >> > > > > >> > ср, 5 февр. 2020 г. в 20:47, Denis Magda <dma...@apache.org>: > > > >> > > > > >> > > Hi Vladimir, > > > >> > > > > > >> > > Yes, I'm suggesting us to enable this flag by > > > >> > > > > > >> > > > > >> > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE > > > >> > > default instead of introducing --force flag and showing any > > > warnings. > > > >> > > > > > >> > > - > > > >> > > Denis > > > >> > > > > > >> > > > > > >> > > On Wed, Feb 5, 2020 at 2:33 AM Vladimir Steshin < > > vlads...@gmail.com > > > > > > > >> > > wrote: > > > >> > > > > > >> > > > Hello all. > > > >> > > > > > > >> > > > Denis, which changes exactly? In current implementation of > > ticket > > > >> [2] > > > >> > > flag > > > >> > > > [1] is checked before requiring --force flag and showing any > > > >> warnings. > > > >> > Do > > > >> > > > we need to set reuse-memory-on-deactivate to true by default? > > > >> > > > > > > >> > > > [1] > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE > > > >> > > > > > > >> > > > [2] https://issues.apache.org/jira/browse/IGNITE-12614 > > > >> > > > > > > >> > > > > > > >> > > > вт, 4 февр. 2020 г. в 22:45, Denis Magda <dma...@apache.org>: > > > >> > > > > > > >> > > > > That's the best solution for this scenario. Should we > readjust > > > the > > > >> > > > already > > > >> > > > > created ticket [1] suggesting to implement the changes of > Alex > > > >> > > Scherbakov > > > >> > > > > instead? > > > >> > > > > > > > >> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12614 > > > >> > > > > > > > >> > > > > - > > > >> > > > > Denis > > > >> > > > > > > > >> > > > > > > > >> > > > > On Mon, Feb 3, 2020 at 11:54 PM Alexei Scherbakov < > > > >> > > > > alexey.scherbak...@gmail.com> wrote: > > > >> > > > > > > > >> > > > > > Folks, > > > >> > > > > > > > > >> > > > > > For a long time we have a flag [1] > > > >> > > > > > > > > >> > > > > > It does almost what we want here. > > > >> > > > > > > > > >> > > > > > I suggest to make this behavior default and rework it to > > keep > > > >> data > > > >> > in > > > >> > > > > > memory as well (we already have special "recovery" mode > for > > > >> > caches). > > > >> > > > > > > > > >> > > > > > [1] > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > org.apache.ignite.IgniteSystemProperties#IGNITE_REUSE_MEMORY_ON_DEACTIVATE > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > пн, 3 февр. 2020 г. в 18:47, Alexey Goncharuk < > > > >> > > > > alexey.goncha...@gmail.com > > > >> > > > > > >: > > > >> > > > > > > > > >> > > > > > > I do not mind making this change if we explicitly and > > > clearly > > > >> > > define > > > >> > > > > what > > > >> > > > > > > 'new inactive state' means. What should happen if a > > > partition > > > >> is > > > >> > > lost > > > >> > > > > in > > > >> > > > > > > inactive state? What if such node joins back the cluster > > > >> after? > > > >> > > Etc. > > > >> > > > > > > > > > >> > > > > > > пт, 31 янв. 2020 г. в 20:57, Denis Magda < > > dma...@apache.org > > > >: > > > >> > > > > > > > > > >> > > > > > > > Back up Ivan's opinion here. Initially, the > > > >> > > activation/deactivation > > > >> > > > > was > > > >> > > > > > > > created for the baseline topology designed for cases > > with > > > >> > native > > > >> > > > > > > > persistence. My thinking was that the mechanism > itended > > to > > > >> > > prevent > > > >> > > > > data > > > >> > > > > > > > inconsistencies while nodes with data on the disk will > > be > > > in > > > >> > the > > > >> > > > > > process > > > >> > > > > > > of > > > >> > > > > > > > joining the cluster. > > > >> > > > > > > > > > > >> > > > > > > > Artem, could you please update the docs bringing this > to > > > the > > > >> > > > > attention > > > >> > > > > > of > > > >> > > > > > > > the user community? > > > >> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-12615 > > > >> > > > > > > > > > > >> > > > > > > > AG, what if we don't purge data from memory at least > for > > > the > > > >> > > caches > > > >> > > > > not > > > >> > > > > > > > backed by the native persistence? Is this a big deal? > We > > > can > > > >> > > > > certainly > > > >> > > > > > > put > > > >> > > > > > > > this off by my guts feel we'll return to this question > > > >> sooner > > > >> > or > > > >> > > > > later. > > > >> > > > > > > > > > > >> > > > > > > > - > > > >> > > > > > > > Denis > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > On Fri, Jan 31, 2020 at 2:17 AM Ivan Pavlukhin < > > > >> > > > vololo...@gmail.com> > > > >> > > > > > > > wrote: > > > >> > > > > > > > > > > >> > > > > > > > > For me it looks like some coincidence effect. I > > > understand > > > >> > that > > > >> > > > we > > > >> > > > > > get > > > >> > > > > > > > > such behavior because deactivation works the same > way > > as > > > >> for > > > >> > > > > > > > > persistent caches. Was cluster > activation/deactivation > > > >> > designed > > > >> > > > and > > > >> > > > > > > > > described for in-memory caches? Current behavior > > sounds > > > >> for > > > >> > me > > > >> > > a > > > >> > > > as > > > >> > > > > > > > > big risk. I expect a lot of upset users unexpectedly > > > >> purged > > > >> > all > > > >> > > > > their > > > >> > > > > > > > > data. > > > >> > > > > > > > > > > > >> > > > > > > > > пт, 31 янв. 2020 г. в 00:00, Alexey Goncharuk < > > > >> > > > > > > > alexey.goncha...@gmail.com > > > >> > > > > > > > > >: > > > >> > > > > > > > > > > > > >> > > > > > > > > > Because originally the sole purpose of > deactivation > > > was > > > >> > > > resource > > > >> > > > > > > > > > deallocation. > > > >> > > > > > > > > > > > > >> > > > > > > > > > чт, 30 янв. 2020 г. в 22:13, Denis Magda < > > > >> > dma...@apache.org > > > >> > > >: > > > >> > > > > > > > > > > > > >> > > > > > > > > > > Such a revelation for me that data is purged > from > > > RAM > > > >> if > > > >> > > > > someone > > > >> > > > > > > > > > > deactivates the cluster. Alex, do you remember > why > > > we > > > >> > > decided > > > >> > > > > to > > > >> > > > > > > > > implement > > > >> > > > > > > > > > > it this way initially? > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > - > > > >> > > > > > > > > > > Denis > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > On Thu, Jan 30, 2020 at 2:09 AM Alexey > Goncharuk < > > > >> > > > > > > > > > > alexey.goncha...@gmail.com> > > > >> > > > > > > > > > > wrote: > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > I agree on CLI and JMX because those > interfaces > > > can > > > >> be > > > >> > > used > > > >> > > > > by > > > >> > > > > > a > > > >> > > > > > > > > system > > > >> > > > > > > > > > > > administrator and can be invoked by mistake. > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > As for the Java API, personally, I find it > > strange > > > >> to > > > >> > add > > > >> > > > > > 'force' > > > >> > > > > > > > or > > > >> > > > > > > > > > > > 'confirm' flags to it because it is very > > unlikely > > > >> that > > > >> > > such > > > >> > > > > an > > > >> > > > > > > > > invocation > > > >> > > > > > > > > > > > is done by mistake. Such mistakes are caught > > > during > > > >> the > > > >> > > > > testing > > > >> > > > > > > > > phase and > > > >> > > > > > > > > > > > developers will end up hard-coding 'true' as a > > > flag > > > >> > > > anyways. > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > -- > > > >> > > > > > > > > Best regards, > > > >> > > > > > > > > Ivan Pavlukhin > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > -- > > > >> > > > > > > > > >> > > > > > Best regards, > > > >> > > > > > Alexei Scherbakov > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > -- > > > > Best regards, > > Alexei Scherbakov > > >
