Random server restarts, swap and moving nodes to new hardware

2011-08-16 Thread Jeff Pollard
Hello everyone, We've got a very interesting problem. We're hosting our 5-node cluster on EC2 running Ubuntu 10.04 LTS (Lucid Lynx) Server 64-bit using m2.xlarge instance types, and over the past 5 days we've had two EC2 servers randomly restart on us. We've chec

Re: Random server restarts, swap and moving nodes to new hardware

2011-08-16 Thread Sean Cribbs
Jeff, We highly recommend you upgrade to 10.10 or later. 10.04 has some known problems when running under Xen (especially on EC2) -- in some cases under load, the network interface will break, making the node temporarily inaccessible. When you do upgrade, the simplest way (if possible) would be t

Re: Random server restarts, swap and moving nodes to new hardware

2011-08-16 Thread Jeff Pollard
Hey Sean, Thanks very much for the reply. I'll certainly try going to 10.10 with the upgrade, that's good info. Re EBS: were you saying to attach the EBS volume to the new node and use the EBS volume as the data volume? We had been using EBS as a backup volume, but using ephemeral storage for t

Re: Random server restarts, swap and moving nodes to new hardware

2011-08-16 Thread Sean Cribbs
Jeff, That should be fine. I assumed you were using EBS as your data volume because most people we talk to do (for better or worse). On Tue, Aug 16, 2011 at 10:32 AM, Jeff Pollard wrote: > Hey Sean, > > Thanks very much for the reply. I'll certainly try going to 10.10 with the > upgrade, that's

Riak memory usage higher than expected

2011-08-16 Thread Jacques
We're utilizing Riak 14.2 and we're seeing higher memory consumption than we expect. We're running on a 4 node cluster with each node housing 32gb of memory and are utilizing bitcask with a 3x write replication factor. We're seeing faster growth than we expect and also seeing weird bounces update