Failed to perform cache operation (cache is stopped)

[Akash Shinde]

Hi,
I have four server nodes and six client nodes on ignite cluster. I am using
ignite 2.10 version.
Some operations are failing due to the CacheStoppedException exception on
the server nodes. This has become a blocker issue.
Could someone please help me to resolve this issue.

*Cache Configuration*

CacheConfiguration subscriptionCacheCfg = new
CacheConfiguration<>(CacheName.SUBSCRIPTION_CACHE.name());
subscriptionCacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
subscriptionCacheCfg.setWriteThrough(false);
subscriptionCacheCfg.setReadThrough(true);
subscriptionCacheCfg.setRebalanceMode(CacheRebalanceMode.ASYNC);
subscriptionCacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
subscriptionCacheCfg.setBackups(*2*);
Factory<SubscriptionDataLoader> storeFactory =
FactoryBuilder.factoryOf(SubscriptionDataLoader.class);
subscriptionCacheCfg.setCacheStoreFactory(storeFactory);
subscriptionCacheCfg.setIndexedTypes(DefaultDataKey.class,
SubscriptionData.class);
subscriptionCacheCfg.setSqlIndexMaxInlineSize(47);
RendezvousAffinityFunction affinityFunction = new RendezvousAffinityFunction();
affinityFunction.setExcludeNeighbors(true);
subscriptionCacheCfg.setAffinity(affinityFunction);
subscriptionCacheCfg.setStatisticsEnabled(true);
subscriptionCacheCfg.setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);


*Exception stack trace*

ERROR c.q.dgms.kafka.TaskRequestListener - Error occurred while consuming
the object
com.baidu.unbiz.fluentvalidator.exception.RuntimeValidateException:
java.lang.IllegalStateException: class
org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed
to perform cache operation (cache is stopped): SUBSCRIPTION_CACHE
at
com.baidu.unbiz.fluentvalidator.FluentValidator.doValidate(FluentValidator.java:506)
at
com.baidu.unbiz.fluentvalidator.FluentValidator.doValidate(FluentValidator.java:461)
at
com.xyz.dgms.service.UserManagementServiceImpl.deleteUser(UserManagementServiceImpl.java:710)
at
com.xyz.dgms.kafka.TaskRequestListener.processRequest(TaskRequestListener.java:190)
at
com.xyz.dgms.kafka.TaskRequestListener.process(TaskRequestListener.java:89)
at
com.xyz.libraries.mom.kafka.consumer.TopicConsumer.lambda$run$3(TopicConsumer.java:162)
at net.jodah.failsafe.Functions$12.call(Functions.java:274)
at net.jodah.failsafe.SyncFailsafe.call(SyncFailsafe.java:145)
at net.jodah.failsafe.SyncFailsafe.run(SyncFailsafe.java:93)
at
com.xyz.libraries.mom.kafka.consumer.TopicConsumer.run(TopicConsumer.java:159)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: class
org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed
to perform cache operation (cache is stopped): SUBSCRIPTION_CACHE
at
org.apache.ignite.internal.processors.cache.GridCacheGateway.enter(GridCacheGateway.java:166)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.onEnter(GatewayProtectedCacheProxy.java:1625)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.get(GatewayProtectedCacheProxy.java:673)
at
com.xyz.dgms.grid.dao.AbstractDataGridDAO.getData(AbstractDataGridDAO.java:39)
at
com.xyz.dgms.grid.dao.AbstractDataGridDAO.getData(AbstractDataGridDAO.java:28)
at
com.xyz.dgms.grid.dataservice.DefaultDataGridService.getData(DefaultDataGridService.java:22)
at
com.xyz.dgms.grid.dataservice.DefaultDataGridService.getData(DefaultDataGridService.java:10)
at
com.xyz.dgms.validators.common.validators.UserDataValidator.validateSubscription(UserDataValidator.java:226)
at
com.xyz.dgms.validators.common.validators.UserDataValidator.validateRequest(UserDataValidator.java:124)
at
com.xyz.dgms.validators.common.validators.UserDataValidator.validate(UserDataValidator.java:346)
at
com.xyz.dgms.validators.common.validators.UserDataValidator.validate(UserDataValidator.java:41)
at
com.baidu.unbiz.fluentvalidator.FluentValidator.doValidate(FluentValidator.java:490)
... 12 common frames omitted
Caused by:
org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed
to perform cache operation (cache is stopped): SUBSCRIPTION_CACHE
... 24 common frames omitted


Thanks,
Akash