Now reading about ZFS, and see that it includes RAID-Z, which sounds very interesting indeed. Don't know how to thank you guys for all the quality feedback I am getting through this list. I greatly appreciate it.
-- Mads Nordholm On Tue, Mar 3, 2015 at 8:20 PM, Mads Nordholm <[email protected]> wrote: > Very useful input indeed. I think I might end up going with a more > conventional setup for starters, and then play with CEPH on the site. And > that then leads to another question: Does anybody have some input on what > RAID level to use for a more conventional storage setup? I am looking at > deploying a setup that exclusively uses SSD, so I am probably a bit more > interested in getting as many usable GBs as possible, than I am in > optimising I/O. > > So far, I have been hearing people advocating RAID 10 as well as RAID 6. I > am personally leaning towards RAID 6, but I would love to get some input > from someone with more experience using these different RAID levels in > production. > > -- > Mads Nordholm > > On Tue, Mar 3, 2015 at 7:34 PM, Vadim Kimlaychuk < > [email protected]> wrote: > >> Andrija, >> >> This is my choise already -- FreeBSD + ZFS with SSD for ZIL/L2ARC >> cache + NFS. Going to be at production within couple of weeks. You have >> read my thoughts ! :) >> >> Vadim. >> >> -----Original Message----- >> From: Andrija Panic [mailto:[email protected]] >> Sent: Tuesday, March 03, 2015 2:25 PM >> To: [email protected] >> Subject: Re: Hardware question >> >> I'm personaly having fights with CEPh used for Primary storage - I ike >> CEPH VERY MUCH, but hate it at the same time (hars word, I know...) >> >> For Primary storage - my suggestions, play arround if you like, but avoid >> it at the end...till it matures better, or simply the integration with CEPH >> matures better. >> >> If you are not using 10G network and serious hardware - it's crappy >> experience... SSD for Journal, etc... >> >> It's a fight - whenever I do some maintance on CEPH I end up swetting, >> clients asking why is everythgin so slow, etc... >> >> For our next cloud, I'm going with ZFS/NFS definitively... >> >> Be warned :) >> >> Cheers >> >> On 3 March 2015 at 13:15, Vadim Kimlaychuk <[email protected]> >> wrote: >> >> > Mads, >> > >> > CEPH is good indeed, but keep in mind that you should really >> > be expert at this type of SDS. There are points that are not visible >> > from the first look and may bring some unpleasent surprises. For >> example: "default" >> > option for storage I have tested was to make snapshots automatically >> > from the files being saved to primary storage. As a consequence when >> > you delete VM there are artifacts (snapshots) that are connect to >> > deleted VM not being deleted by Cloudstack (since CS does not know they >> exist). >> > Another point - you can't directly use it as secondary >> > storage. Need to set-up application server and run RadosGW. >> > Performance - is a big question mark here. You need NFS or iSCSI anyway. >> > What we haven't fully tested - disaster recovery or >> > malfunction simulation. You must know how to recover from all types of >> > the faults. It is very easy to lose everything by just doing wrong >> > things (or in wrong order). From my point of view Ceph is rather >> > complex to start together with CS. It may be easy to set up, but not so >> easy to manage. >> > Will suggest you to run it like a year at development to make >> > yourself confident you can manage it. >> > >> > Regards, >> > >> > Vadim. >> > >> > -----Original Message----- >> > From: Mads Nordholm [mailto:[email protected]] >> > Sent: Monday, March 02, 2015 8:16 PM >> > To: [email protected] >> > Subject: Re: Hardware question >> > >> > Thanks a lot for your answer, Lucian. CEPH sounds like a very >> > interesting solution. I will have to do some more research on that. >> > >> > -- >> > Mads Nordholm >> > >> > On Tue, Mar 3, 2015 at 12:32 AM, Nux! <[email protected]> wrote: >> > >> > > Hi Mads, >> > > >> > > Imo, if you want that flexibility you should go with non-local >> storage. >> > > CEPH is a popular choice here, but you will need 10 Gbps between >> > > hypervisors and storage servers if you want reasonable performance. >> > > So, if you need more storage just add more CEPH servers. Need more >> > > compute, add more hypervisors. >> > > >> > > HTH >> > > Lucian >> > > >> > > -- >> > > Sent from the Delta quadrant using Borg technology! >> > > >> > > Nux! >> > > www.nux.ro >> > > >> > > ----- Original Message ----- >> > > > From: "Mads Nordholm" <[email protected]> >> > > > To: [email protected] >> > > > Sent: Monday, 2 March, 2015 17:19:40 >> > > > Subject: Hardware question >> > > >> > > > I am planning a small Cloudstack setup (using KVM for >> > > > virtualisation) >> > > that >> > > > will allow me to run roughly 100 VPSs with these average >> requirements: >> > > > >> > > > - 1 core >> > > > - 512 MB RAM >> > > > - 20 GB SSD >> > > > >> > > > I am interested in input regarding a hardware configuration that >> > > > will support this, and how to best build a small setup that will >> > > > scale easily >> > > as >> > > > I grow. Within a year or so, I expect to have more than 1,000 >> > > > guest >> > > running. >> > > > >> > > > I basically need a setup that will not completely break the bank >> > > > as I >> > > start >> > > > out, but also one that will scale well as I grow. I am >> > > > particularly concerned with being able to add only the resources I >> > > > need. If I need >> > > more >> > > > storage, I want to be able to add only that (preferably just by >> > > > adding disks to a RAID array), and if I need more computing power, >> > > > I want to be able to add only that. >> > > > >> > > > Any input greatly appreciated. >> > > > >> > > > -- >> > > > Mads Nordholm >> > > >> > >> >> >> >> -- >> >> Andrija Panić >> > >
