Not sure if there is a particular reason for you using different regions, but Amazon states that each zone is a different physical location completely separate from others, e.g. us-east-1a and us-east-1b. Using the Amazon internal IPs (10.x. etc) reduces latency greatly by not going outbound through DNS (though us-east-1c is twice as latent), you have an RF 4 possible (in different physical locations!), and most importantly.... sh*t just works.
Maybe this won't help you, but it may be useful for others :) On Thu, Feb 24, 2011 at 10:53 AM, Dave Viner <davevi...@gmail.com> wrote: > Another possibility is this: > > why not setup 2 nodes in 1 region in 1 az, and get that to work. > Then, open a third node in the same region, but different AZ, and get that > to work. > Then, once you have that working, open a fourth node in a different region > and get that to work. > > Seems like taking a piece-meal approach would be beneficial here. > > Dave Viner > > > On Thu, Feb 24, 2011 at 6:11 AM, Daniel van Ham Colchete < > daniel.colch...@gmail.com> wrote: > >> Himanshi, >> >> my bad, try this for iptables: >> >> # SNAT outgoing connections >> iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j >> SNAT --to-source INTERNALIP >> >> As for tcpdump the argument for the -i option is the interface name (eth0, >> cassth0, etc...), and not the IP. So, it should be >> tcpdump -i cassth0 -n port 7000 >> or >> tcpdump -i eth0 -n port 7000 >> >> I`m assuming your main network card is eth0, but that should be the case. >> >> Does it work? >> >> Best, >> Daniel >> >> >> On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma <himanshi.sha...@tcs.com >> > wrote: >> >>> >>> Thanks Daniel. >>> >>> But SNAT command is not working and when i try tcpdump it gives >>> >>> [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000 >>> tcpdump: Invalid adapter index >>> >>> Not able to figure out wats this ?? >>> >>> Thanks, >>> Himanshi >>> >>> >>> >>> From: Daniel van Ham Colchete <daniel.colch...@gmail.com> To: >>> user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: >>> Cassandra nodes on EC2 in two different regions not communicating >>> ------------------------------ >>> >>> >>> >>> Himanshi, >>> >>> you could try adding your public IP address to an internal interface and >>> DNAT the packets to it. This shouldn't give you any problems with your >>> normal traffic. Tell Cassandra on listen on the public IPs and it should >>> work. >>> >>> Linux commands would be: >>> >>> # Create an internal interface using bridge-utils >>> brctl addbr cassth0 >>> >>> # add the ip >>> ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32> >>> >>> # DNAT incoming connections >>> iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT >>> --to-destination 50.18.60.117 >>> >>> # SNAT outgoing connections >>> iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT >>> --to-source INTERNALIP >>> >>> This should work since Amazon you re-SNAT your outgoing packets to your >>> public IP again, so the other cassandra instance will see your public IP as >>> your source address. >>> >>> I didn't test this setup here but it should work unless I forgot some >>> small detail. If you need to troubleshoot use the command "tcpdump -i >>> INTERFACE -n port 7000" where INTERFACE should be your public interface or >>> your cassth0. >>> >>> Please let me know if it worked. >>> >>> Best regards, >>> Daniel Colchete >>> >>> On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <* >>> himanshi.sha...@tcs.com* <himanshi.sha...@tcs.com>> wrote: >>> giving private ip to rpc address gives the same exception >>> and the keeping it blank and providing public to listen also fails. I >>> tried keeping both blank and did telnet on 7000 so i get following o/p >>> >>> [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000 >>> Trying 122.248.193.37... >>> Connected to 122.248.193.37. >>> Escape character is '^]'. >>> >>> Similarly from another achine >>> >>> [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000 >>> Trying 184.72.22.87... >>> Connected to 184.72.22.87. >>> Escape character is '^]'. >>> >>> >>> >>> -----Dave Viner wrote: ----- >>> To: *user@cassandra.apache.org* <user@cassandra.apache.org> >>> From: Dave Viner <*davevi...@gmail.com* <davevi...@gmail.com>> >>> Date: 02/24/2011 11:59AM >>> cc: Himanshi Sharma <*himanshi.sha...@tcs.com* <himanshi.sha...@tcs.com> >>> > >>> >>> Subject: Re: Cassandra nodes on EC2 in two different regions not >>> communicating >>> >>> Try using the private ipv4 address in the rpc_address field, and the >>> public ipv4 (NOT the elastic ip) in the listen_address. >>> >>> If that fails, go back to rpc_address empty, and start up cassandra. >>> >>> Then from the other node, please telnet to port 7000 on the first node. >>> And show the output of that session in your reply. >>> >>> I haven't actually constructed a cross-region cluster nor have I used >>> v0.7, but this really sounds like it should be easy. >>> >>> On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >>> * <himanshi.sha...@tcs.com>> wrote: >>> Hi Dave, >>> >>> I tried with the public ips. If i mention the public ip in rpc address >>> field, Cassandra gives the same exception but if leave it blank then >>> Cassandra runs but again in the nodetool command with ring option it does'nt >>> show the node in another region. >>> >>> Thanks, >>> Himanshi >>> >>> >>> -----Dave Viner wrote: ----- >>> To: *user@cassandra.apache.org * <user@cassandra.apache.org> >>> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> >>> Date: 02/24/2011 10:43AM >>> >>> Subject: Re: Cassandra nodes on EC2 in two different regions not >>> communicating >>> >>> That looks like it's not an issue of communicating between nodes. It >>> appears that the node can not bind to the address on the localhost that >>> you're asking for. >>> >>> " java.net.BindException: Cannot assign requested address " >>> >>> I think the issue is that the Elastic IP address is not actually an IP >>> address that's on the localhost. So the daemon can not bind to that IP. >>> Instead of using the EIP, use the local IP address for the rpc_address (i >>> think that's what you need since that is what Thrift will bind to). Then >>> for the listen_address should be the ip address that is routable from the >>> other node. I would first try with the actual public IP address (not the >>> Elastic IP). Once you get that to work, then shutdown the cluster, change >>> the listen_address to the EIP, boot up and try again. >>> >>> Dave Viner >>> >>> >>> On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >>> * <himanshi.sha...@tcs.com>> wrote: >>> >>> Hey Dave, >>> >>> Sorry i forgot to mention the Non-seed configuration. >>> >>> for first node in us-west its as below....i.e its own elastic ip >>> >>> listen_address: 50.18.60.117 >>> rpc_address: 50.18.60.117 >>> >>> and for second node in ap-southeast-1 its as below....i.e again its own >>> elastic ip >>> >>> listen_address: 175.41.143.192 >>> rpc_address: 175.41.143.192 >>> >>> Thanks, >>> Himanshi >>> >>> >>> >>> >>> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> To: >>> *user@cassandra.apache.org * <user@cassandra.apache.org> Date: 02/23/2011 >>> 11:01 PM Subject: Re: Cassandra nodes on EC2 in two different regions >>> not communicating >>> >>> ------------------------------ >>> >>> >>> >>> internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., >>> from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east >>> to us-west). To do regions, you must use the public ip address assigned by >>> amazon. >>> >>> Himanshi, when you log into 1 node, and telnet to port 7000 on the other >>> node, which IP address did you use - the 10.x address or the public ip >>> address? >>> And what is the seed/non-seed configuration in both cassandra.yaml files? >>> >>> >>> Dave Viner >>> >>> >>> On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < *fr...@isidorey.com >>> *<fr...@isidorey.com>> >>> wrote: >>> The internal Amazon IP address is what you will want to use so you don't >>> have to go through DNS anyways; not sure if this works from US-East to >>> US-West, but it does make things quicker in between zones, e.g. us-east-1a >>> to us-east-1b. >>> >>> >>> On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < *davevi...@gmail.com >>> *<davevi...@gmail.com>> >>> wrote: >>> Try using the IP address, not the dns name in the cassandra.yaml. >>> >>> If you can telnet from one to the other on port 7000, and both nodes have >>> the other node in their config, it should work. >>> >>> Dave Viner >>> >>> >>> On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < *himanshi.sha...@tcs.com >>> * <himanshi.sha...@tcs.com>> wrote: >>> >>> Ya they do. Have specified Public DNS in seed field of each node in >>> Cassandra.yaml...nt able to figure out what the problem is ??? >>> >>> >>> From: Sasha Dolgy < *sdo...@gmail.com * <sdo...@gmail.com>> To: >>> *user@cassandra.apache.org >>> * <user@cassandra.apache.org> Date: 02/23/2011 02:56 PM Subject: Re: >>> Cassandra nodes on EC2 in two different regions not communicating >>> >>> >>> ------------------------------ >>> >>> >>> >>> did you define the other host in the cassandra.yaml ? on both servers >>> .... they need to know about each other >>> >>> On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < *himanshi.sha...@tcs.com >>> * <himanshi.sha...@tcs.com>> wrote: >>> >>> Thanks Dave but I am able to telnet to other instances on port 7000 >>> and when i run ./nodetool --host >>> *ec2-50-18-60-117.us-west-1.compute.amazonaws.com >>> * <http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> ring... I >>> can see only one node. >>> >>> Do we need to configure anything else in Cassandra.yaml or >>> Cassandra-env.sh ??? >>> >>> >>> >>> >>> From: Dave Viner < *davevi...@gmail.com * <davevi...@gmail.com>> To: >>> *user@cassandra.apache.org * <user@cassandra.apache.org> Cc: Himanshi >>> Sharma < *himanshi.sha...@tcs.com * <himanshi.sha...@tcs.com>> Date: >>> 02/23/2011 >>> 11:36 AM Subject: Re: Cassandra nodes on EC2 in two different regions >>> not communicating >>> >>> >>> >>> ------------------------------ >>> >>> >>> >>> If you login to one of the nodes, can you telnet to port 7000 on the >>> other node? >>> >>> If not, then almost certainly it's a firewall/Security Group issue. >>> >>> You can find out the security groups for any node by logging in, and then >>> running: >>> >>> % curl " *http://169.254.169.254/latest/meta-data/security-groups >>> *<http://169.254.169.254/latest/meta-data/security-groups>" >>> >>> >>> Assuming that both nodes are in the same security group, ensure that the >>> SG is configured to allow other members of the SG to communicate on port >>> 7000 to each other. >>> >>> HTH, >>> Dave Viner >>> >>> >>> On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma < *himanshi.sha...@tcs.com >>> * <himanshi.sha...@tcs.com>> wrote: >>> >>> Hi, >>> >>> I am new to Cassandra. I m running Cassandra on EC2. I configured >>> Cassandra cluster on two instances in different regions. >>> But when I am trying the nodetool command with ring option, I am getting >>> only single node. >>> >>> How to make these two nodes communicate with each other. I have already >>> opened required ports. i.e 7000, 8080, 9160 in respective >>> security groups. Plz help me with this. >>> >>> Regards, >>> Himanshi Sharma >>> >>> >>> =====-----=====-----===== >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have received this communication in error, >>> >>> please notify us by reply e-mail or telephone and >>> immediately and permanently delete the message >>> and any attachments. Thank you >>> >>> >>> >>> >>> =====-----=====-----===== >>> >>> >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> >>> >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have received this communication in error, >>> please notify us by reply e-mail or telephone and >>> immediately and permanently delete the message >>> >>> >>> and any attachments. Thank you >>> >>> >>> >>> >>> >>> -- >>> Sasha Dolgy * >>> **sasha.do...@gmail.com * <sasha.do...@gmail.com> >>> >>> =====-----=====-----===== >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> >>> >>> >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have received this communication in error, >>> >>> >>> >>> please notify us by reply e-mail or telephone and >>> immediately and permanently delete the message >>> and any attachments. Thank you >>> >>> >>> >>> >>> >>> >>> -- >>> Frank LoVecchio >>> Senior Software Engineer | Isidorey, LLC >>> Google Voice +1.720.295.9179 * >>> **isidorey.com * <http://isidorey.com/>| *facebook.com/franklovecchio >>> *<http://facebook.com/franklovecchio>| >>> *franklovecchio.com * <http://franklovecchio.com/> >>> >>> >>> >>> =====-----=====-----===== >>> >>> Notice: The information contained in this e-mail >>> >>> message and/or attachments to it may contain >>> >>> confidential or privileged information. If you are >>> >>> >>> not the intended recipient, any dissemination, use, >>> >>> review, distribution, printing or copying of the >>> >>> information contained in this e-mail message >>> >>> and/or attachments to it are strictly prohibited. If >>> >>> you have received this communication in error, >>> >>> >>> please notify us by reply e-mail or telephone and >>> >>> immediately and permanently delete the message >>> >>> and any attachments. Thank you >>> >>> >>> >>> >>> =====-----=====-----===== >>> >>> Notice: The information contained in this e-mail >>> >>> message and/or attachments to it may contain >>> >>> confidential or privileged information. If you are >>> >>> >>> not the intended recipient, any dissemination, use, >>> >>> review, distribution, printing or copying of the >>> >>> information contained in this e-mail message >>> >>> and/or attachments to it are strictly prohibited. If >>> >>> you have received this communication in error, >>> >>> >>> please notify us by reply e-mail or telephone and >>> >>> immediately and permanently delete the message >>> >>> and any attachments. Thank you >>> >>> >>> >>> >>> =====-----=====-----===== >>> >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> >>> >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have received this communication in error, >>> >>> >>> please notify us by reply e-mail or telephone and >>> immediately and permanently delete the message >>> and any attachments. Thank you >>> >>> >>> >>> >>> =====-----=====-----===== >>> >>> Notice: The information contained in this e-mail >>> message and/or attachments to it may contain >>> confidential or privileged information. If you are >>> >>> >>> >>> not the intended recipient, any dissemination, use, >>> review, distribution, printing or copying of the >>> information contained in this e-mail message >>> and/or attachments to it are strictly prohibited. If >>> you have received this communication in error, >>> >>> >>> >>> please notify us by reply e-mail or telephone and >>> immediately and permanently delete the message >>> and any attachments. Thank you >>> >>> >>> >> > -- Frank LoVecchio Senior Software Engineer | Isidorey, LLC Google Voice +1.720.295.9179 isidorey.com | facebook.com/franklovecchio | franklovecchio.com
