Hi, So how should I handle that exception? Should I just ignore it? Is there a way I can know when the data streamer successfully deliver after getting that error? I just want to know if data loading/rebalancing finished succesfully.
On Fri, Apr 21, 2017 at 2:10 PM, Evgenii Zhuravlev <[email protected]> wrote: > Hi, > > Minor topology version changes when you creating or removing caches and at > late affinity. > > Yes, you see this error in log because minor topoly was changed while > datastreamer was running, but you didn't lose any data. > > According to javadoc: > >>Note that there is no guarantee that data will be delivered after this >> concrete attempt (e.g., it can fail when topology is changing), but it won't >> be lost anyway. > > Evgenii > > > 2017-04-21 13:24 GMT+03:00 Alper Tekinalp <[email protected]>: >> >> Hi. >> >> First of all what is minor topology version and when does it chage? >> >> Below we got an error while loading data with data streamer: >> >> 18/Apr/2017 13:50:45 INFO 34391455 [exchange-worker-#31%null%] >> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) - >> Started cache [name=RECORD_CACHE_XX, mode=PARTITIONED] >> .. >> 18/Apr/2017 13:50:49 ERROR 34394884 [DeploymentWorker-0] >> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper(L:359) - Scenario >> data could not be preloaded in distributed deployment for scenario [XX]. >> javax.cache.CacheException: class >> org.apache.ignite.IgniteCheckedException: Failed to finish operation (too >> many remaps): 32 >> at >> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1465) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1160) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1244) >> at >> com.intellica.evam.engine.db.dao.ScenarioRecordDao.preloadScenarioRecords(ScenarioRecordDao.java:134) >> at >> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper.preloadScenarioRecordData(PreloadHelper.java:58) >> at >> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper.preloadScenarioData(PreloadHelper.java:24) >> at >> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper.distributedDeploy(DeploymentHelper.java:827) >> at >> com.intellica.evam.engine.dynamic.DeploymentWorker.handleDeploymentEvent(DeploymentWorker.java:83) >> at >> com.intellica.evam.engine.dynamic.DeploymentWorker.run(DeploymentWorker.java:62) >> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to >> finish operation (too many remaps): 32 >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:863) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:828) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:456) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:439) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:259) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:389) >> at >> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:355) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1789) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:335) >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1215) >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:843) >> at >> org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108) >> at >> org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:783) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer >> request failed [node=6fe302a3-f353-4235-af5a-3708bf240750] >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1777) >> ... 8 more >> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer >> will retry data transfer at stable topology [reqTop=AffinityTopologyVersion >> [topVer=11, minorTopVer=14], topVer=AffinityTopologyVersion [topVer=11, >> minorTopVer=15], node=remote] >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:339) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:297) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:56) >> at >> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:86) >> ... 7 more >> 18/Apr/2017 13:50:49 INFO 34394950 [exchange-worker-#31%null%] >> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) - >> Started cache [name=FEC_XX, mode=PARTITIONED] >> >> >> I guess the reason seems that when one of the server nodes loads data >> through data streamer to cache RECORD_CACHE_XX, other server creates cache >> FEC_XX. Bacause cache creation increases the minor topology version. >> >> Is my understanding true? >> If it is what is the reason? >> >> -- >> Alper Tekinalp >> >> Software Developer >> Evam Streaming Analytics >> >> Atatürk Mah. Turgut Özal Bulv. >> Gardenya 5 Plaza K:6 Ataşehir >> 34758 İSTANBUL >> >> Tel: +90 216 455 01 53 Fax: +90 216 455 01 54 >> www.evam.com.tr > > -- Alper Tekinalp Software Developer Evam Streaming Analytics Atatürk Mah. Turgut Özal Bulv. Gardenya 5 Plaza K:6 Ataşehir 34758 İSTANBUL Tel: +90 216 455 01 53 Fax: +90 216 455 01 54 www.evam.com.tr
