On 13 March 2018 at 04:54, Kenneth Brotman <kenbrot...@yahoo.com.invalid> wrote:
> Kunal, > > > > Please provide the following setting from the yaml files you are using: > > > > seeds: > In GCE: seeds: "10.142.14.27" In AWS (new node being added): seeds: "35.196.96.247,35.227.127.245,35.196.241.232" (these are the public IP addresses of 3 nodes from GCE) I have verified that I am able to do cqlsh from the AWS instance to all 3 ip addresses. > listen_address: > We use the listen_interface setting instead of listen_address. In GCE: listen_interface: eth0 (running ubuntu 14.04 LTS) In AWS: listen_interface: ens3 (running ubuntu 16.04 LTS) > broadcast_address: > I tried setting broadcast_address to one instance in GCE: broadcast_address: 35.196.96.247 In AWS: broadcast_address: 13.127.89.251 (this is the public/elastic IP of the node in AWS) rpc_address: > Like listen_address, we use rpc_interface. In GCE: rpc_interface: eth0 In AWS: rpc_interface: ens3 > endpoint_snitch: > In both setups, we currently use GossipingPropertyFileSnitch. The cassandra-rackdc.properties files from both setups: GCE: dc=DC1 rack=RAC1 AWS: dc=DC2 rack=RAC1 > auto_bootstrap: > When the google cloud instances started up, we hadn't set this explicitly - so, they started off with default value (auto_bootstrap: true) However, as outlined in the datastax doc for adding new dc, I had added 'auto_bootstrap: false' to the google cloud instances (not restarted the service as per the doc). In the AWS instance, I had added 'auto_bootstrap: false' - the doc says we need to do "nodetool rebuild" and hence no automatic bootstrapping. But, haven't gotten to that step yet. Thanks, Kunal > > Kenneth Brotman > > > > *From:* Kunal Gangakhedkar [mailto:kgangakhed...@gmail.com] > *Sent:* Monday, March 12, 2018 4:13 PM > *To:* user@cassandra.apache.org > *Cc:* Nikhil Soman > *Subject:* Re: [EXTERNAL] RE: Adding new DC? > > > > > > On 13 March 2018 at 00:06, Durity, Sean R <sean_r_dur...@homedepot.com> > wrote: > > You cannot migrate and upgrade at the same time across major versions. > Streaming is (usually) not compatible between versions. > > > > I'm not trying to upgrade as of now - first priority is the migration. > > We can look at version upgrade later on. > > > > > > As to the migration question, I would expect that you may need to put the > external-facing ip addresses in several places in the cassandra.yaml file. > And, yes, it would require a restart. Why is a non-restart more desirable? > Most Cassandra changes require a restart, but you can do a rolling restart > and not impact your application. This is fairly normal admin work and > can/should be automated. > > > > I just tried setting the broadcast_address in one of the instances in GCE > to its public IP and restarted the service. > > However, it now shows all other nodes (in GCE) as DN in nodetool status > output and the other nodes also report this node as DN with its > internal/private IP address. > > > > I also tried setting the broadcast_rpc_address to the internal/private IP > address - still the same. > > > > > > How large is the cluster to migrate (# of nodes and size of data). The > preferred method might depend on how much data needs to move. Is any > application outage acceptable? > > > > No. of nodes: 5 > > RF: 3 > > Data size (as reported by the load factor in nodetool status output): > ~30GB per node > > > > Thanks, > Kunal > > > > > > Sean Durity > > lord of the (C*) rings (Staff Systems Engineer – Cassandra) > > *From:* Kunal Gangakhedkar [mailto:kgangakhed...@gmail.com] > *Sent:* Sunday, March 11, 2018 10:20 PM > *To:* user@cassandra.apache.org > *Subject:* [EXTERNAL] RE: Adding new DC? > > > > Hi Kenneth, > > > > Replies inline below. > > > > On 12-Mar-2018 3:40 AM, "Kenneth Brotman" <kenbrot...@yahoo.com.invalid> > wrote: > > Hi Kunal, > > > > That version of Cassandra is too far before me so I’ll let others answer. > I was wonder why you wouldn’t want to end up on 3.0x if you’re going > through all the trouble of migrating anyway? > > > > > > Application side constraints - some data types are different between 2.1.x > and 3.x (for example, date vs. timestamp). > > > > Besides, this is production setup - so, cannot take risk. > > Are both data centers in the same region on AWS? Can you provide yaml > file for us to see? > > > > > > No, they are in different regions - GCE setup is in us-east while AWS > setup is in Asia-south (Mumbai) > > > > Thanks, > > Kunal > > Kenneth Brotman > > > > *From:* Kunal Gangakhedkar [mailto:kgangakhedkar@gmailcom > <kgangakhed...@gmail.com>] > *Sent:* Sunday, March 11, 2018 2:32 PM > *To:* user@cassandra.apache.org > *Subject:* Adding new DC? > > > > Hi all, > > > > We currently have a cluster in GCE for one of the customers. > > They want it to be migrated to AWS. > > > > I have setup one node in AWS to join into the cluster by following: > > https://docs.datastax.com/en/cassandra/2.1/cassandra/ > operations/ops_add_dc_to_cluster_t.html > <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_cassandra_2.1_cassandra_operations_ops-5Fadd-5Fdc-5Fto-5Fcluster-5Ft.html&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=4s2PNt4_Ty1RVe_0dQ4sn-jQTjmz-Wmxnf2OS4URoYo&s=pfA6Jkn2UwG7AISlAM3OJ1OzQpghd_nVJj-KnYLCvBk&e=> > > > > Will add more nodes once the first one joins successfully. > > > > The node in AWS has an elastic IP - which is white-listed for ports > 7000-7001, 7199, 9042 in GCE firewall. > > > > The snitch is set to GossipingPropertyFileSnitch. The GCE setup has > dc=DC1, rack=RAC1 while on AWS, I changed the DC to dc=DC2. > > > > When I start cassandra service on the AWS instance, I see the version > handshake msgs in the logs trying to connect to the public IPs of the GCE > nodes: > > OutboundTcpConnection.java:496 - Handshaking version with /xx.xx.xx.xx > > However, nodetool status output on both sides don't show the other side at > all. That is, the GCE setup doesn't show the new DC (dc=DC2) and the AWS > setup doesn't show old DC (dc=DC1). > > > > In cassandra.yaml file, I'm only using listen_interface and rpc_interface > settings - no explicit IP addresses used - so, ends up using the internal > private IP ranges. > > > > Do I need to explicitly add the broadcast_address? for both side? > > Would that require restarting of cassandra service on GCE side? Or is it > possible to change that setting on-the-fly without a restart? > > > > I would prefer a non-restart option. > > > > PS: The cassandra version running in GCE is 2.1.18 while the new node > setup in AWS is running 2.1.20 - just in case if that's relevant > > > > Thanks, > > Kunal > > > > > ------------------------------ > > > The information in this Internet Email is confidential and may be legally > privileged. It is intended solely for the addressee. Access to this Email > by anyone else is unauthorized. If you are not the intended recipient, any > disclosure, copying, distribution or any action taken or omitted to be > taken in reliance on it, is prohibited and may be unlawful. When addressed > to our clients any opinions or advice contained in this Email are subject > to the terms and conditions expressed in any applicable governing The Home > Depot terms of business or client engagement letter. The Home Depot > disclaims all responsibility and liability for the accuracy and content of > this attachment and for any damages or losses arising from any > inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other > items of a destructive nature, which may be contained in this attachment > and shall not be liable for direct, indirect, consequential or special > damages in connection with this e-mail message or its attachment. > > >
