Re: Cross data center HA cluster

2022-05-26 Thread Iliya Grushevskiy
Hi, Aaron We are currently testing similar deployment and have encountered several issues: - message lose on send on network failure between data centers I think this is due to the fact that HA replication is asynchronous and replica server may not catch up with primary. - message lose or dup

RE: [EXTERNAL]:Re: Cross data center HA cluster

2022-05-26 Thread Steigerwald, Aaron
Hello Iliya, Thank you very much for you response, it's very helpful. Regarding "message loss on send on network failure between data centers"- the example architecture I described does not have master/slave HA pairs in separate data centers. Do you think the message loss you described has anyt

Re: [EXTERNAL]:Re: Cross data center HA cluster

2022-05-26 Thread Илья Грушевский
I have a simple test with single HA pair in which I just kill master while sending messages in different thread. (it almost identical to replicated-transaction-failover example, except the different thread thing) And I encounter the same message loss, as in network failure scenario. I suspect the

Re: [EXTERNAL]:Re: Cross data center HA cluster

2022-05-26 Thread Iliya Grushevskiy
Hi Aaron. Sorry there were some misleading information in my previous message. I have reviewed my test and it does contains network failure. If I turn off network failure everything works as expected. So as I understand if you consider your LAN a reliable network there should not be any messag

Re: Cross data center HA cluster

2022-05-26 Thread Justin Bertram
> I think this is due to the fact that HA replication is asynchronous and replica server may not catch up with primary. To be clear, message replication between a primary and a backup is *synchronous*. Justin On Thu, May 26, 2022 at 3:11 AM Iliya Grushevskiy wrote: > Hi, Aaron > > We are curr

Re: Cross data center HA cluster

2022-05-26 Thread Илья Грушевский
You are right, I should have not use the term asynchronous. But in case of replication failure, for example network failure it will not fail a transaction. So if I gradually lose connections, first between primary and backup and then between primary and client I will lose all send message between

Re: Artemis- Using Netty http tunnel for SAAS service

2022-05-26 Thread Justin Bertram
> I tried running a sample JMS program using Netty Http tunnelling to connect to the Artemis service via the public interface but the connection fails... Can you provide the source code and configuration for this sample JMS program? > Server side I am seeing some error after some time (not immedi

Re: Cross data center HA cluster

2022-05-26 Thread Justin Bertram
> But in case of replication failure, for example network failure it will not fail a transaction. I'm not exactly sure what you mean here. Are you saying that if a primary is replicating to a backup and a client is in the middle of a transaction and the replication connection between the primary a

Re: Cross data center HA cluster

2022-05-26 Thread Илья Грушевский
Justin, thanks for replying. Now I’ve got the point of such behavior. Yes, that is exactly what I meant. If you expect any connection loss between HA pair while having client able to connect to both, you may encounter message loss, and replication is not a right solution. > 27 мая 2022 г., в 01

Re: Cross data center HA cluster

2022-05-26 Thread Justin Bertram
> If you expect any connection loss between HA pair while having client able to connect to both, you may encounter message loss, and replication is not a right solution. Can you elaborate on where the message loss would be in your example here? Please be precise about the details as they are espec

Re: Cross data center HA cluster

2022-05-26 Thread Iliya Grushevskiy
Suppose we have a cluster. And we have a HA pair of primary (P) and backup (B) nodes in this cluster. 1. Client connects to P and start sending messages 2. Network between P and B fails 3. Client continue sending message to P 4. Quorum vote completes. B become active and P stops All messages bet