Hi Team, Issue is fixed. There was issue with my TcpDiscoveryStaticIpFinder setup.
Regards Satyajit From: satyajit.mandal.barclays.com via user <user@ignite.apache.org> Sent: Tuesday, May 9, 2023 10:05 AM To: user@ignite.apache.org Subject: RE: .NET Ignite server nodes not joining cluster using TcpDiscoveryStaticIpFinder CAUTION: This email originated from outside our organisation - user@ignite.apache.org<mailto:user@ignite.apache.org> Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe. Hi Team, Any idea why this TCP connection error happening. We haven't seen these errors while running Ignite from Java binaries. This is first time we are running from binaries generated from .NET nuget package and hosting Ignite as webApi service ( .NET server node). The second node when we start it starts giving TCP errors . However looking at the logs it seems cluster is getting formed but this error keeps coming back. Have seen some similar post where second node using static ip is not able to join the cluster and throwing errors. https://stackoverflow.com/questions/46065368/apache-ignite-net-server-node-hangs-on-start<https://clicktime.symantec.com/15siFAJHJNeLrDaNgF3te?h=0P4WG2RhT4pSU6DIBFQjZz4i3tLEIRYHwuKQsn9XDws=&u=https://stackoverflow.com/questions/46065368/apache-ignite-net-server-node-hangs-on-start> Regards Satyajit From: Mandal, Satyajit: IT (PUN) Sent: Monday, May 8, 2023 8:41 PM To: 'user@ignite.apache.org' <user@ignite.apache.org<mailto:user@ignite.apache.org>>; 'ptupit...@apache.org' <ptupit...@apache.org<mailto:ptupit...@apache.org>> Subject: RE: .NET Ignite server nodes not joining cluster using TcpDiscoveryStaticIpFinder Hi Pavel, We are seeing lot of interim TCP communication errors and node leaving and joining the cluster when we hosted Ignite as WebApi service. Its rejoining immediately. Can we conclude that this could be because Ignite is embedded with .NET host ? [15:44:18,930][SEVERE][exchange-worker-#46%ignite-instance-0612556c-26f7-4f30-b224-cc85a4e80e3f%][diagnostic] Failed to send diagnostic message: class o.a.i.IgniteCheckedException: Failed to send message (node may have left the grid or TCP connection cannot be established due to firewall issues) [node=TcpDiscoveryNode [id=aba2e264-40e4-4281-9c20-b4b2c86b9a72, consistentId=10.115.179.174:47500, addrs=ArrayList [10.115.179.174], sockAddrs=HashSet [GBRPSM030001102.INTRANET.BARCAPINT.com/10.115.179.174:47500], discPort=47500, order=2, intOrder=2, lastExchangeTime=1683555768564, loc=false, ver=2.14.0#20220929-sha1:951e8deb, isClient=false], topic=TOPIC_INTERNAL_DIAGNOSTIC, msg=IgniteDiagnosticMessage [flags=1, futId=7], policy=2] [15:44:18,931][INFO][grid-nio-worker-tcp-comm-3-#26%TcpCommunicationSpi%][diagnostic] Exchange future on coordinator waiting for server response [node=aba2e264-40e4-4281-9c20-b4b2c86b9a72, topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0]] Regards Satyajit From: Mandal, Satyajit: IT (PUN) Sent: Sunday, May 7, 2023 5:19 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org>; Mandal, Satyajit: IT (PUN) <satyajit.man...@barclays.com<mailto:satyajit.man...@barclays.com>>; ptupit...@apache.org<mailto:ptupit...@apache.org> Subject: .NET Ignite server nodes not joining cluster using TcpDiscoveryStaticIpFinder Hi Pavel/Team, Issue is resolved and cluster is getting formed. I corrected the TCPDiscoverySpi code and it worked.In LocalAddresses provided local node IP and under TcpDiscoveryStaticIpFinder added remote node IP and current IP. Regards Satyajit From: satyajit.mandal.barclays.com via user <user@ignite.apache.org<mailto:user@ignite.apache.org>> Sent: Saturday, May 6, 2023 8:42 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org>; ptupit...@apache.org<mailto:ptupit...@apache.org> Subject: RE: .NET Ignite server nodes not joining cluster using TcpDiscoveryStaticIpFinder CAUTION: This email originated from outside our organisation - user@ignite.apache.org<mailto:user@ignite.apache.org> Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe. Hi Pavel/Team, I tried to remove the Spring.xml and wrote everything using Ignite .NET Api. Looks like TcpDiscoveryStaticIpFinder is not working as expected when we host host Ignite as .NET webApi windows service. I hosted this webapi in two machines as windows service and started the same as server node. It only runs with local host and cluster is not getting formed. Logs are moving but nodes are not joining into cluster. They are running as separate instance. @Pavel: Can you please confirm if we host ignite as .NET webapi windows service then this TcpDiscoveryStaticIpFinder won't work? This is a major blocker for our project? Appreciate if we get quick response. services.AddOptions<IgniteConfiguration>().Configure<IConfiguration>((opt, cfg) => { cfg.Bind("Ignite", opt); opt.DiscoverySpi = new TcpDiscoverySpi { LocalPort = 47500, LocalPortRange = 5, ForceServerMode = true, LocalAddress = "127.0.0.1", IpFinder = new TcpDiscoveryStaticIpFinder { Endpoints = new[] { "127.0.0.1:47500", "10.115.179.175:47500..47505", "10.113.48.148:47500..47505", } } }; opt.IsActiveOnStart = true; opt.Localhost = cfg.GetValue<string>("Ignite:LocalHost"); opt.JvmOptions = new[] { cfg.GetValue<string>("Ignite:JVMOptions:DIgnite_Quiet"), cfg.GetValue<string>("Ignite:JVMOptions:DIgnite_Performance_Suggestions_disabled"), cfg.GetValue<string>("Ignite:JVMOptions:Xms"), cfg.GetValue<string>("Ignite:JVMOptions:Xmx"), cfg.GetValue<string>("Ignite:JVMOptions:Djava.net.preferIPv4Stack") }; opt.PeerAssemblyLoadingMode = cfg.GetValue<PeerAssemblyLoadingMode>("Ignite:PeerAssemblyLoadingMode"); opt.AutoGenerateIgniteInstanceName = cfg.GetValue<bool>("Ignite:AutoGenerateIgniteInstanceName"); opt.DataStorageConfiguration = new DataStorageConfiguration { DefaultDataRegionConfiguration = new DataRegionConfiguration { Name = "Default_Region", InitialSize = 100 * 1024 * 1024 }, DataRegionConfigurations = new[] { new DataRegionConfiguration { Name = "Department", InitialSize = 20 * 1024 * 1024, MaxSize = 40 * 1024 * 1024, PageEvictionMode = DataPageEvictionMode.Random2Lru }, new DataRegionConfiguration { Name = "Person", InitialSize = 15 * 1024 * 1024, MaxSize = 30 * 1024 * 1024, }, new DataRegionConfiguration { Name = "TradeSubType", InitialSize = 15 * 1024 * 1024, MaxSize = 30 * 1024 * 1024, } } }; }); services.AddSingleton<IIgnite>(sp => { var config = sp.GetRequiredService<IOptions<IgniteConfiguration>>(); var ignite = Ignition.Start(config.Value); ignite.GetCluster().SetActive(true); return ignite; }); Regards Satyajit From: satyajit.mandal.barclays.com via user <user@ignite.apache.org<mailto:user@ignite.apache.org>> Sent: Friday, May 5, 2023 4:58 PM To: ptupit...@apache.org<mailto:ptupit...@apache.org>; user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Ignite nodes not joining cluster and logs not moving CAUTION: This email originated from outside our organisation - user@ignite.apache.org<mailto:user@ignite.apache.org> Do not click on links, open attachments, or respond unless you recognize the sender and can validate the content is safe. Hi Pavel, We have deployed two instances of .NET WebApi in two servers and hosted the same as windows service. We started both the instances running in two different machine. However we see that on startup they are not joining the cluster and ignite logs are not moving after initial logging. Have attached the ignite log and spring.xml which we were using . Could you please check what is missing. >From code we have put these properties. "Ignite": { "LocalHost": "127.0.0.1", "SpringConfigUrl": "C:/SpringConfig/IgniteConfig.xml", "JVMOptions": { "DIgnite_Quiet": "-DIGNITE_QUIET=true", "DIgnite_Performance_Suggestions_disabled": "-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true", "Xms": "-Xms6g", "Xmx": "-Xmx6g" }, "AutoGenerateIgniteInstanceName": true, "PeerAssemblyLoadingMode": "CurrentAppDomain" }, This webapi works fine if we put 127.0.0.1 in spring.xml and with single instance. Regards Satyajit Barclays Execution Services Limited registered in England. Registered No. 1767980. Registered office: 1 Churchill Place, London, E14 5HP Barclays Execution Services Limited provides support and administrative services across Barclays group. Barclays Execution Services Limited is an appointed representative of Barclays Bank UK plc, Barclays Bank plc and Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays Bank plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. Clydesdale Financial Services Limited is authorised and regulated by the Financial Conduct Authority. This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender and immediately delete it and any attachments from your system. Do not copy, use, disclose or otherwise act on any part of this email or its attachments. Internet communications are not guaranteed to be secure or virus-free. The Barclays group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays group for operational or business reasons. Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays group is personal to the sender and is not given or endorsed by the Barclays group. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Barclays Execution Services Limited registered in England. Registered No. 1767980. Registered office: 1 Churchill Place, London, E14 5HP Barclays Execution Services Limited provides support and administrative services across Barclays group. Barclays Execution Services Limited is an appointed representative of Barclays Bank UK plc, Barclays Bank plc and Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays Bank plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. Clydesdale Financial Services Limited is authorised and regulated by the Financial Conduct Authority. This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender and immediately delete it and any attachments from your system. Do not copy, use, disclose or otherwise act on any part of this email or its attachments. Internet communications are not guaranteed to be secure or virus-free. The Barclays group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays group for operational or business reasons. Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays group is personal to the sender and is not given or endorsed by the Barclays group. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Barclays Execution Services Limited registered in England. Registered No. 1767980. Registered office: 1 Churchill Place, London, E14 5HP Barclays Execution Services Limited provides support and administrative services across Barclays group. Barclays Execution Services Limited is an appointed representative of Barclays Bank UK plc, Barclays Bank plc and Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays Bank plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. Clydesdale Financial Services Limited is authorised and regulated by the Financial Conduct Authority. This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender and immediately delete it and any attachments from your system. Do not copy, use, disclose or otherwise act on any part of this email or its attachments. Internet communications are not guaranteed to be secure or virus-free. The Barclays group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays group for operational or business reasons. Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays group is personal to the sender and is not given or endorsed by the Barclays group. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Barclays Execution Services Limited registered in England. Registered No. 1767980. Registered office: 1 Churchill Place, London, E14 5HP Barclays Execution Services Limited provides support and administrative services across Barclays group. Barclays Execution Services Limited is an appointed representative of Barclays Bank UK plc, Barclays Bank plc and Clydesdale Financial Services Limited. Barclays Bank UK plc and Barclays Bank plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and the Prudential Regulation Authority. Clydesdale Financial Services Limited is authorised and regulated by the Financial Conduct Authority. This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender and immediately delete it and any attachments from your system. Do not copy, use, disclose or otherwise act on any part of this email or its attachments. Internet communications are not guaranteed to be secure or virus-free. The Barclays group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays group for operational or business reasons. Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays group is personal to the sender and is not given or endorsed by the Barclays group. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays.
