Re: advice for EC2 deployment

2011-06-23 Thread Sasha Dolgy
we use a combination of Vyatta & OpenVPN on the nodes that are EC2 and nodes that aren't Ec2works a treat. On Thu, Jun 23, 2011 at 10:23 PM, Sameer Farooqui wrote: > EC2Snitch doesn't currently support multi-Regions in Amazon. > Tickets to track: > https://issues.apache.org/jira/browse/CASSAN

Re: advice for EC2 deployment

2011-06-23 Thread Sameer Farooqui
. > > has anybody got there nodes talking to each other across regions by just > using public-dns? > > I am also looking into open vpn and how to deploy it. > > -- > View this message in context: > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/advice-for-EC2-d

Re: advice for EC2 deployment

2011-06-23 Thread pankajsoni0126
ss regions by just using public-dns? I am also looking into open vpn and how to deploy it. -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/advice-for-EC2-deployment-tp6294613p6508278.html Sent from the cassandra-u...@incubator.apache.org ma

Re: advice for EC2 deployment

2011-06-23 Thread Sasha Dolgy
are you able to open a connection from one of the nodes to a node on the other side? us-east to us-west? could your problem be as simple as connectivity and/or security group configuration? On Thu, Jun 23, 2011 at 1:51 PM, pankaj soni wrote: > hey, > I have got my ec2 multi-dc across AZ's but i

Re: advice for EC2 deployment

2011-06-23 Thread pankaj soni
hey, I have got my ec2 multi-dc across AZ's but in same region us-east. Now I am trying to deploy cassandra over multiple regions that is ec2 us west, singapore and us-east. I have edited the config file as sasha's reply below. though when I run nodetool in each DC, I only see the nodes from tha

Re: advice for EC2 deployment

2011-04-28 Thread aaron morton
If you are not going to be multi-region straight away, but wish to be in the near future I would consider: - 1 region - 2 AZ's, with the same number of nodes - Using the EC2Snitch as is, this will map to 1 cassandra DC and 2 cassandra Racks - Using the NetworkTopology strategy For background s

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
I think you're right about changing NetworkToplogyStrategy, but the timing isn't working in my favor at this point. I wonder how bad that will really be On Wed, Apr 27, 2011 at 9:35 AM, Sasha Dolgy wrote: > so can you not simply leverage a strategy that replicates data between > "racks" and

Re: advice for EC2 deployment

2011-04-27 Thread Sasha Dolgy
so can you not simply leverage a strategy that replicates data between "racks" and at some point in the future when you move to multi-dc upgrade the replication strategy to maintain the current replication and add in some replication between DC's ... ? i'll go re-read your posts to see if you've a

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
Oh, and Route53 doesn't do anything automatically, but there is an API to manage the DNS. It's up to you to run a task on instance boot/terminate, or a cron job if you want to do this trick (for now, seems like a solid future feature of Route53). Though, I hear geographical aware Route53 is alrea

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
I don't think of it as migrating an instance, it's more of a destroy/start with EC2. But, I still think it would be very useful to spin up a set of instances with known hostnames (cassandra1, 2, 3... N) and be able to quickly SSH to them by doing "ssh ec2u...@cassandra1.random.ec2.mydomain.com ".

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
Thanks Sasha. Fortunately/unfortunately I did realize the default & current behavior of the Ec2Snitch, but my application isn't multi-region capable (yet), so I need to get intra-region redundancy. And having a SingleRegionEc2Snitch that did DC=ec2zone and RACK=??? would be much better for me (fo

Re: advice for EC2 deployment

2011-04-27 Thread Sasha Dolgy
if you migrate the instance, does Route53 automatically re-map all the information to the new ec2 instance? another issue is that cassandra only maintains the IP of the other nodes, and not the hostname (assumed based on output of the nodetool ring) ... which means, if you migrate the instance a

Re: advice for EC2 deployment

2011-04-27 Thread Sasha Dolgy
Hi William, The default behavior of Ec2Snitch is outlined below: http://svn.apache.org/repos/asf/cassandra/trunk/src/java/org/apache/cassandra/locator/Ec2Snitch.java // Split "us-east-1a" or "asia-1a" into "us-east"/"1a" and "asia"/"1a". String azone = new String(b ,"UTF-8");

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
It's great advice, but I'm still torn. I've never done multi-region work before, and I'd prefer to wait for 0.8 with built-in inter-node security, but I'm otherwise ready to roll (and need to roll) cassandra out sooner than that. Given how well my system held up with a total single AZ failure, I'

Re: advice for EC2 deployment

2011-04-27 Thread William Oberman
While I haven't configured it for multi-region yet, Sasha is exactly right now how amzon's DNS works (returning private vs. public IP depending on if the machine is local to the region or not). For extra fun, now that Route53 exists you can (somewhat trivially) map and dynamically maintain all EC2

Re: advice for EC2 deployment

2011-04-27 Thread Sasha Dolgy
Hi, If I understand you correctly, you are trying to get a private ip in us-east speaking to the private ip in us-west. to make your life easier, configure your nodes to use hostname of the server. if it's in a different region, it will use the public ip (ec2 dns will handle this for you) and if

Re: advice for EC2 deployment

2011-04-27 Thread pankajsoni0126
's for such a scenario in EC2, as Public Ip are less secure and costly? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/advice-for-EC2-deployment-tp6294613p6309154.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.

Re: advice for EC2 deployment

2011-04-27 Thread pankajsoni0126
's for such a scenario in EC2, as Public Ip are less secure and costly? -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/advice-for-EC2-deployment-tp6294613p6309153.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.

Re: advice for EC2 deployment

2011-04-27 Thread aaron morton
Using the EC2Snitch you could have one AZ in us-east-1 and one Az in us-west-1, treat each AZ as a single rack and each region as a DC. The network topology is rack aware so will prefer request that go to the same rack (not much of an issue when you have only one rack). If possible I would use

Re: advice for EC2 deployment

2011-04-26 Thread William Oberman
I see what you're saying. I was able to control write latency on mysql using insert vs insert delayed (what I feel is MySQLs poor man's eventual consistency option) + the fact that replication was a background asynchronous process. In terms of read latency, I was able to do up to a few hundred we

Re: advice for EC2 deployment

2011-04-26 Thread aaron morton
One difference between Cassandra and MySQL replication may be when the network IO happens. Was the MySQL replication synchronous on transaction commit ? I was only aware that it had async replication, which means the client is not exposed to the network latency. In cassandra the network latency

Re: advice for EC2 deployment

2011-04-26 Thread William Oberman
Thanks Aaron! Unless no one on this list uses EC2, there were a few minor troubles end of last week through the weekend which taught me a lot about obscure failure modes in various applications I use :-) My original post was trying to be more redundant than fast, which has been by overall goal fr

Re: advice for EC2 deployment

2011-04-25 Thread aaron morton
For background see this article: http://www.datastax.com/dev/blog/deploying-cassandra-across-multiple-data-centers And this recent discussion http://www.mail-archive.com/user@cassandra.apache.org/msg12502.html Issues that may be a concern: - lots of cross AZ latency in us-east, e.g. LOCAL_QUORUM

advice for EC2 deployment

2011-04-21 Thread William Oberman
Hi, My service is not yet ready to be fully multi-DC, due to how some of my legacy MySQL stuff works. But, I wanted to get cassandra going ASAP and work towards multi-DC. I have two main cassandra use cases: one where I can handle eventual consistency (and all of the writes/reads are currently O