Am 13.01.2011 23:17, schrieb Jonathan Tripathy: > > On 13/01/11 21:34, Stan Hoeppner wrote: >> Jonathan Tripathy put forth on 1/13/2011 7:11 AM: >> >>> Would DRBD + GFS2 work better than NFS? While NFS is simple, I don't >>> mind >>> experimenting with DRBD and GFS2 is it means fewer problems? >> Depends on your definition of "better". If you do two dovecot+drbd >> nodes you >> have only two nodes. If you do NFS you have 3 including the NFS server. >> Performance would be very similar between the two. >> >> Now, when you move to 3 dovecot nodes or more you're going to run into >> network >> scaling problems with the drbd traffic, because it increases >> logarithmically (or >> is it exponentially?) with node count. If using GFS2 atop drbd across >> all >> nodes, each time a node writes to GFS, the disk block gets >> encapsulated by the >> drbd driver and transmitted to all other drbd nodes. With each new >> mail that's >> written by each server, or each flag is updated, it gets written 4 >> times, once >> locally, and 3 times via drbd. >> >> With NFS, each of these writes occurs over the network only once. >> With drbd >> it's always a good idea to dedicate a small high performance GbE >> switch to the >> cluster nodes just for drbd traffic. This may not be necessary in a >> low volume >> environment, but it's absolutely necessary in high traffic setups. >> Beyond a >> certain number of nodes even in a moderately busy mail network, drbd >> mirroring >> just doesn't work. The bandwidth requirements become too high, and >> nodes bog >> down from processing all of the drbd packets. Without actually using >> it myself, >> and just using some logical reasoning based on the technology, I'd say >> the ROI >> of drbd mirroring starts decreasing rapidly between 2 and 4 nodes, and >> beyond >> for nodes... >> >> You'd be much better off with an NFS server, or GFS2 directly on a SAN >> LUN. >> CXFS would be far better, but it's not free. In fact it's rather >> expensive, and >> it requires a dedicated metadata server(s), which is one of the >> reasons it's so >> #@! damn fast compared to most clustered filesystems. >> >> Another option is a hybrid setup, with dual NFS servers each running GFS2 >> accessing the shared SAN LUN(s). This eliminates the one NFS server as a >> potential single point of failure, but also increases costs >> significantly as you >> have to spend about $15K USD minimum for low end SAN array, and >> another NFS >> server box, although the latter need not be expensive. >> > Hi Stan, > > The problem is, is that we do not have the budget at the minute to buy a > SAN box, which is why I'm just looking to setup Linux environment to > substitute for now. > > Regarding the servers, I was thinking of having a 2 node drbd cluster > (in active+standby), which would export a single iSCSI LUN. Then, I > would have a 2 node dovecot+postfix cluster (in active-active), where > each node would mount the same LUN (With GFS2 on top). This is 4 servers > in total (Well, 4 VMs running on 4 physically separate servers). > > I'm hearing different things on whether dovecot works well or not with > GFS2. Of course, I could simply replace the iSCSI LUN above with an nfs > server running on each DRBD node, if you feel NFS would work better than > GFS2. Either way, I would probably use a crossover cable for the DRBD > cluster. Could maybe even bond 2 cables together if I'm feeling > adventurous! > > The way I see it, is that there are 2 issues to deal with: > > 1) Which "Shared Disk" technology is best (GFS2 over LUN or a simple NFS > server) > and > 2) What is the best method of HA for the storage system > > Any advice is appreciated.
Hi Jonathan the explains from Stan were good enough to choose what fit to your needs ( thx Stan for explain drbd so deeply ) so what are the number of mailboxes and the traffic volume you wait for ? ) at minimum you should have 2 drbd nodes binding to a seperate gb interface via cross cable ( this might not be needed with virtual machines, but check before you setup, and dont forget for real ha you always need 2 vm master machines, so for your setup this may increase the budget), and 2 loadblancers with i.e keepalive, if this isnt enough for you, you should follow Stans advice using SAN or equal, after all this is the real world , budget must always be as high to solve your task, you cant press an elephant trough a mouse hole.... so there is no best solution, there is only a solution what fits best to your needs -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
