- the AWS people say EIPs are a PITA. - if we hardcode the global IPs in the yaml, then yaml editing is required for the occaisional hard instance reboot in aws and its attendant global ip reassignment - if we try leaving broadcast_rpc_address blank, null , or commented out with rpc_address set to 0.0.0.0 then cassandra refuses to start - if we take out rpc_address and broadcast_rpc_address, then cqlsh doesn't work with localhost anymore and that fucks up some of our cluster managemetn tooling
- we kind of are being lazy and just want what worked in 2.1 to work in 2.2 Ok, the code in 2.1: https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java Of interest DatabaseDescriptor.setBroadcastAddress(localPublicAddress); DatabaseDescriptor.setBroadcastRpcAddress(localPublicAddress); The code in 2.2+: https://github.com/apache/cassandra/blob/cassandra-2.2/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java <https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/locator/Ec2MultiRegionSnitch.java> Becomes DatabaseDescriptor.setBroadcastAddress(localPublicAddress); if (DatabaseDescriptor.getBroadcastRpcAddress() == null) { logger.info("broadcast_rpc_address unset, broadcasting public IP as rpc_address: {}", localPublicAddress); DatabaseDescriptor.setBroadcastRpcAddress(localPublicAddress); } And that if clause added as part of a CASSANDRA-11356 patch, is what is submarining us. I don't know the otherwise intricacies of the various address settings in the yaml vis-a-vis EC2MRS, but since we can't configure it the good-old-2.1-way in 2.2+, this seems broken to us. I'll try to track down where cassandra startup is complaining to us about rpc_address: 0.0.0.0 and broadcast_rpc_address being blank/null/commented out. That section of code may need an exception for EC2MRS. On Tue, Mar 26, 2019 at 12:01 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Tue, Mar 26, 2019 at 5:49 PM Carl Mueller > <carl.muel...@smartthings.com.invalid> wrote: > >> Looking at the code it appears it shouldn't matter what we set the yaml >> params to. The Ec2MultiRegionSnitch should be using the aws metadata >> 169.254.169.254 to pick up the internal/external ips as needed. >> > > This is somehow my expectation as well, so maybe the docs are just > outdated. > > I think I'll just have to dig in to the code differences between 2.1 and >> 2.2. We don't want to specify the glboal IP in any of the yaml fields >> because the global IP for the instance changes if we do an aws instance >> restart. Don't want yaml editing to be a part of the instance restart >> process. >> > > We did solve this in the past by using Elastic IPs: anything prevents you > from using those? > > -- > Alex > >
