Hello! Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING
Sounds pretty informative ^ No, REPLICATED caches will not replicate values loaded from the cache store. It is assumed that it may be fetched transparently from underlying store. For REPLICATED cache there's nothing to rebalance when node leaves. Regards, -- Ilya Kasnacheev чт, 3 июн. 2021 г. в 11:06, xmw45688 <xw...@procurant.com>: > Hi Ignitians, > > I fail to understand what causes and need your help - > 1) When k8s sees “Critical system error”, it will restart ignite-admin > server. Restarting is fine because of the critical system error. But what > are the causes of the critical system error? > 2) Critical system error may be corresponding to JVM held. Still we > don’t > know the reason why JVM got held. > 3) The cluster lost one ignite client node, probably due to OOME > 4) Why/how was ignite server node triggered to reload the data from > Cassandra (all data in C* tables are cached once the ignite server starts. > All SQL DML interacts with Ignite Cache which interact with Cassandra for > insert/update/delete) If Ignite needs to rebalance the data among the > server nodes, why can't rebalance the data from one node to another? if > even rebalancing data, why submitting invalid queries? We are using > apache-ignite-2.8.0.20190215. > > Exceptions - > > [2021-06-02 > 17:09:04,005][ERROR][sys-#103562%ignite-procurant-admin-cluster%][root] > Critical system error detected. Will be handled accordingly to configured > handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, > super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet > [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], > failureCtx=FailureContext [type=CRITICAL_ERROR, err=class > o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Committing a > transaction has produced runtime exception]] > ... 1 more > at > com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:104) > class > org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: > Committing a transaction has produced runtime exception > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.heuristicException(IgniteTxAdapter.java:800) > at > > org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.commitRemoteTx(GridDistributedTxRemoteAdapter.java:847) > at > > org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.commitIfLocked(GridDistributedTxRemoteAdapter.java:795) > at > > org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.salvageTx(GridDistributedTxRemoteAdapter.java:898) > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.salvageTx(IgniteTxManager.java:398) > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.access$3100(IgniteTxManager.java:134) > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$NodeFailureTimeoutObject.onTimeout0(IgniteTxManager.java:2551) > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$NodeFailureTimeoutObject.access$3300(IgniteTxManager.java:2505) > at > > org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager$NodeFailureTimeoutObject$1.run(IgniteTxManager.java:2624) > at > > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6898) > at > > org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: > Cannot > execute this query as it might involve data filtering and thus may have > unpredictable performance. If you want to execute this query despite the > performance unpredictability, use ALLOW FILTERING > at java.lang.Thread.run(Thread.java:745) > at > > org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.load(GridCacheStoreManagerAdapter.java:293) > at > > org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadFromStore(GridCacheStoreManagerAdapter.java:338) > Caused by: class org.apache.ignite.IgniteCheckedException: class > org.apache.ignite.IgniteException: Failed to execute Cassandra CQL > statement: select "id", "sourceid", "versioning", "colname", "colnewvalue", > "cololdvalue", "createdby", "ts", "op" from "admin"."user_history" where > "id"=?; > at > > org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerReload(GridCacheMapEntry.java:984) > ... 13 more > at > > org.apache.ignite.internal.processors.cache.GridCacheMapEntry.readThrough(GridCacheMapEntry.java:619) > at > > org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.commitIfLocked(GridDistributedTxRemoteAdapter.java:701) > ... 18 more > at > > org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.execute(CassandraSessionImpl.java:198) > Caused by: class org.apache.ignite.IgniteException: Failed to execute > Cassandra CQL statement: select "id", "sourceid", "versioning", "colname", > "colnewvalue", "cololdvalue", "createdby", "ts", "op" from > "admin"."user_history" where "id"=?; > Caused by: javax.cache.integration.CacheLoaderException: class > org.apache.ignite.IgniteException: Failed to execute Cassandra CQL > statement: select "id", "sourceid", "versioning", "colname", "colnewvalue", > "cololdvalue", "createdby", "ts", "op" from "admin"."user_history" where > "id"=?; > ... 17 more > at > > org.apache.ignite.cache.store.cassandra.CassandraCacheStore.load(CassandraCacheStore.java:189) > Caused by: class org.apache.ignite.IgniteException: Failed to execute > Cassandra CQL statement: select "id", "sourceid", "versioning", "colname", > "colnewvalue", "cololdvalue", "createdby", "ts", "op" from > "admin"."user_history" where "id"=?; > at > > org.apache.ignite.cache.store.cassandra.session.WrappedSession.prepare(WrappedSession.java:56) > at > > org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.execute(CassandraSessionImpl.java:180) > ... 20 more > at > > org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.prepareStatement(CassandraSessionImpl.java:665) > Caused by: class org.apache.ignite.IgniteException: Failed to prepare > Cassandra CQL statement: select "id", "sourceid", "versioning", "colname", > "colnewvalue", "cololdvalue", "createdby", "ts", "op" from > "admin"."user_history" where "id"=?; > at > > org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.execute(CassandraSessionImpl.java:147) > at > > com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50) > at > > com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) > Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: > Cannot > execute this query as it might involve data filtering and thus may have > unpredictable performance. If you want to execute this query despite the > performance unpredictability, use ALLOW FILTERING > at > com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:104) > at > > org.apache.ignite.internal.processors.cache.CacheStoreBalancingWrapper.load(CacheStoreBalancingWrapper.java:98) > at > > org.apache.ignite.cache.store.cassandra.session.WrappedSession.prepare(WrappedSession.java:56) > ... 21 more > Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: > Cannot > execute this query as it might involve data filtering and thus may have > unpredictable performance. If you want to execute this query despite the > performance unpredictability, use ALLOW FILTERING > at > > org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadFromStore(GridCacheStoreManagerAdapter.java:327) > at com.datastax.driver.core.Responses$Error.asException(Responses.java:148) > at com.datastax.driver.core.SessionManager$4.apply(SessionManager.java:196) > at com.datastax.driver.core.SessionManager$4.apply(SessionManager.java:220) > at > > com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) > at > > com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) > at > > com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) > at com.google.common.util.concurrent.Futures$2$1.run(Futures.java:1177) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at > > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > > [2021-06-02 > 17:09:04,556][ERROR][sys-#103562%ignite-procurant-admin-cluster%][root] JVM > will be halted immediately due to the failure: [failureCtx=FailureContext > [type=CRITICAL_ERROR, err=class > o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Committing a > transaction has produced runtime exception]] > ... 1 more > at > > org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.prepareStatement(CassandraSessionImpl.java:647) > ... 20 more > [17:09:02] Topology snapshot [ver=7, locNode=103dfcb4, servers=2, > clients=3, > state=ACTIVE, CPUs=15, offheap=6.5GB, heap=8.0GB] > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
