Hi, I've run more detailed tests during the weekend and i can surely tell that problem is not related to the migrated data. With a new cluster setup and with 0 data we can still get the error.
what i have in my mind is this; with the new version there may be a new configuration parameter that has to be set in order cacheentryprocessors to be DEPLOYED in SHARED mode to all cluster nodes, but i cannot find such a parameter. so at this point my problem becomes to this; is there a configuration parameter that forces all cacheentroprocessors the be deployed on every cluster node from client by force? following is the cluster and client configuration; client-server-configs.zip <http://apache-ignite-users.70518.x6.nabble.com/file/t2515/client-server-configs.zip> when the client nodes starts and run necessary jobs -containing cacaheentryprocessors- first 1 or 2 cacheentryprocessors are deploed to both clusters after that new cacaheentryprocessors starts to get classnotfoundexception and cluster nodes keeps giving following warnings; [2021-05-24T15:36:00,010][WARN ][sys-stripe-4-#5][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.ShiftPromotionCountersEntryProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870560005, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=627410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=627410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.ShiftPromotionCountersEntryProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:00,103][WARN ][sys-stripe-2-#3][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870560100, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=a27410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=a27410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:00,180][WARN ][sys-stripe-1-#2][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870560171, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=f27410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=f27410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:00,202][WARN ][sys-stripe-1-#2][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870560191, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=137410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=137410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:00,308][WARN ][sys-stripe-1-#2][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870560302, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=337410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=337410f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreResetProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:16,861][INFO ][grid-timeout-worker-#198][IgniteKernal] Metrics for local node (to disable set 'metricsLogFrequency' to 0) ^-- Node [id=f4e082a1, uptime=00:04:00.184] ^-- Cluster [hosts=8, CPUs=34, servers=2, clients=19, topVer=27, minorTopVer=0] ^-- Network [addrs=[10.240.242.191, 10.240.245.191, 127.0.0.1], discoPort=47500, commPort=47100] ^-- CPU [CPUs=4, curLoad=1.27%, avgLoad=13.55%, GC=0%] ^-- Heap [used=1682MB, free=61.37%, comm=2023MB] ^-- Off-heap memory [used=7427MB, free=71.43%, allocated=25800MB] ^-- Page memory [pages=1879439] ^-- sysMemPlc region [type=internal, persistence=true, lazyAlloc=false, ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.99%, allocRam=100MB, allocTotal=0MB] ^-- default region [type=default, persistence=true, lazyAlloc=true, ... initCfg=256MB, maxCfg=25600MB, usedRam=7427MB, freeRam=70.99%, allocRam=25600MB, allocTotal=13476MB] ^-- metastoreMemPlc region [type=internal, persistence=true, lazyAlloc=false, ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=99.89%, allocRam=0MB, allocTotal=0MB] ^-- TxLog region [type=internal, persistence=true, lazyAlloc=false, ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=100MB, allocTotal=0MB] ^-- volatileDsMemPlc region [type=user, persistence=false, lazyAlloc=true, ... initCfg=40MB, maxCfg=100MB, usedRam=0MB, freeRam=100%, allocRam=0MB] ^-- Ignite persistence [used=13476MB] ^-- Outbound messages queue [size=0] ^-- Public thread pool [active=0, idle=0, qSize=0] ^-- System thread pool [active=1, idle=6, qSize=0] [2021-05-24T15:36:46,778][WARN ][sys-stripe-28-#29][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870606767, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=082510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=082510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:36:51,975][WARN ][sys-stripe-21-#22][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870611965, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=d25510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=d25510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:37:00,011][WARN ][sys-stripe-4-#5][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.ShiftPromotionCountersEntryProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870620009, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=717510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=717510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.ShiftPromotionCountersEntryProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:37:03,124][WARN ][sys-stripe-39-#40][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870623120, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=b38510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=b38510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] [2021-05-24T15:37:10,256][WARN ][sys-stripe-4-#5][GridDeploymentPerVersionStore] Failed to load peer class (ignore if class got undeployed during preloading) [alias=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, dep=SharedDeployment [rmv=false, super=GridDeployment [ts=1621870630246, depMode=SHARED, clsLdr=GridDeploymentClassLoader [id=f4a510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, singleNode=false, nodeLdrMap=HashMap {db22a85d-37a5-45c4-ae63-bdd535eaca44=75f020f9971-db22a85d-37a5-45c4-ae63-bdd535eaca44}, p2pTimeout=5000, usrVer=0, depMode=SHARED, quiet=false], clsLdrId=f4a510f9971-f4e082a1-4012-4720-bcbb-e438359221e1, userVer=0, loc=false, sampleClsName=com.segmentify.lotr.frodo.cacheentryprocessor.RockScoreUpdateProcessor, pendingUndeploy=false, undeployed=false, usage=0]]] what i need is to deploy all cacheentryprocessors to clusters either gracefully or by force. all cacheentryprocessors are called as in this service (i had to remove some part of it due to company regulations); service.zip <http://apache-ignite-users.70518.x6.nabble.com/file/t2515/service.zip> thanks ----- İbrahim Halil Altun Senior Software Engineer @ Segmentify -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/