Hi, I deployed a service from a client node to our grid using the following code:
IgniteCluster cluster = ignite.cluster(); ClusterGroup group = cluster.forAttribute(...); Ignite.services(workerGroup).deployClusterSingleton("blaze/hsbc") It is fine most of the time. However we just encountered a case where we got an exception when some logic tried to use this service: 2018-08-02 16:27:57.713 processors.task.GridTaskWorker [sys-#29%mlog%] ERROR - Failed to obtain remote job result policy for result from ComputeTask.result(..) method (will fail the whole task): GridJobResultImpl [job=C2 [c=ServiceProxyCallable [mtdName=execute, svcName=blaze/hsbc, ignite=null]], sib=GridJobSiblingImpl [sesId=f66f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, nodeId=236a47e9-7fdb-464e-be44-b24d0942d75c, isJobDone=false], jobCtx=GridJobContextImpl [jobId=076f54be461-65c907a3-8fcf-4ddd-acb1-6553be3d1dc9, timeoutObj=null, attrs={}], node=TcpDiscoveryNode [id=236a47e9-7fdb-464e-be44-b24d0942d75c, addrs=[10.23.8.165], sockAddrs=[zhkdlp1712.int.clsa.com/10.23.8.165:0], discPort=0, order=37, intOrder=27, lastExchangeTime=1533148447088, loc=false, ver=2.3.0#20180518-sha1:02cf6abf, isClient=true], ex=class o.a.i.IgniteException: Service not found: blaze/hsbc, hasRes=true, isCancelled=false, isOccupied=true] org.apache.ignite.IgniteException: Remote job threw user exception (override or implement ComputeTask.result(..) method if you would like to have automatic failover for this exception). at org.apache.ignite.compute.ComputeTaskAdapter.result(ComputeTaskAdapter.java:101) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1047) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskWorker$5.apply(GridTaskWorker.java:1040) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6663) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskWorker.result(GridTaskWorker.java:1040) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:858) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1066) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1301) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) [liquid-logic.jar:2.0.10] at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090) [liquid-logic.jar:2.0.10] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] Caused by: org.apache.ignite.IgniteException: Service not found: blaze/hsbc at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1858) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913) ~[liquid-logic.jar:2.0.10] ... 7 more Caused by: org.apache.ignite.internal.processors.service.GridServiceNotFoundException: Service not found: blaze/hsbc at org.apache.ignite.internal.processors.service.GridServiceProxy$ServiceProxyCallable.call(GridServiceProxy.java:408) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.closure.GridClosureProcessor$C2.execute(GridClosureProcessor.java:1855) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker$2.call(GridJobWorker.java:566) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6631) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:560) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:489) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1181) ~[liquid-logic.jar:2.0.10] at org.apache.ignite.internal.processors.job.GridJobProcessor$JobExecutionListener.onMessage(GridJobProcessor.java:1913) ~[liquid-logic.jar:2.0.10] ... 7 more And that's the end of my stack trace (nothing can be found after the ... 7 more.) The above exception got propagated back to the client which deployed the service. Do you know of any reason for this to happen, like why a deployed service will disappear from the grid? And is there any event I can use to detect this? Thanks, Calvin Calvin KL Wong Sr. Lead Engineer, Execution Services D +852 2600 7983 | M +852 9267 9471 | T +852 2600 8888 5/F, One Island East, 18 Westlands Road, Island East, Hong Kong [:1. Social Media Icons:CLSA_Social Media Icons_linkedin.png]<https://hk.linkedin.com/company/clsa>[:1. Social Media Icons:CLSA_Social Media Icons_twitter.png]<https://twitter.com/clsainsights?lang=en>[:1. Social Media Icons:CLSA_Social Media Icons_youtube.png]<https://www.youtube.com/channel/UC0qWp_lLnOcRYmBlCNQgZKA>[:1. Social Media Icons:CLSA_Social Media Icons_facebook.png]<https://www.facebook.com/clsacommunity/> clsa.com<https://www.clsa.com/> Insights. Liquidity. Capital. [CLSA_RGB]<https://www.clsa.com/member> A CITIC Securities Company The content of this communication is intended for the recipient and is subject to CLSA Legal and Regulatory Notices. These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon request. Please consider before printing. CLSA is ISO14001 certified and committed to reducing its impact on the environment.