On Jun 15, 2010, at 4:42 AM, Arve Paalsrud wrote: > Hi, > > We are currently building a storage box based on OpenSolaris/Nexenta using > ZFS. > Our hardware specifications are as follows: > > Quad AMD G34 12-core 2.3 GHz (~110 GHz) > 10 Crucial RealSSD (6Gb/s) > 42 WD RAID Ed. 4 2TB disks + 6Gb/s SAS expanders > LSI2008SAS (two 4x ports) > Mellanox InfiniBand 40 Gbit NICs > 128 GB RAM > > This setup gives us about 40TB storage after mirror (two disks in spare), > 2.5TB L2ARC and 64GB Zil, all fit into a single 5U box. > > Both L2ARC and Zil shares the same disks (striped) due to bandwidth > requirements. Each SSD has a theoretical performance of 40-50k IOPS on 4k > read/write scenario with 70/30 distribution. Now, I know that you should have > mirrored Zil for safety, but the entire box are synchronized with an active > standby on a different site location (18km distance - round trip of 0.16ms + > equipment latency). So in case the Zil in Site A takes a fall, or the > motherboard/disk group/motherboard dies - we still have safety. > > DDT requirements for dedupe on 16k blocks should be about 640GB when main > pool are full (capacity). > > Without going into details about chipsets and such, do any of you on this > list have any experience with a similar setup and can share with us your > thoughts, do's and dont's, and any other information that could be of help > while building and configuring this? > > What I want to achieve is 2 GB/s+ NFS traffic against our ESX clusters (also > InfiniBand-based), with both dedupe and compression enabled in ZFS.
In general, both dedup and compression gain space by trading off performance. You should take a closer look at snapshots + clones because they gain performance by trading off systems management. You can't size by ESX server, because ESX works (mostly) as a pass-through of the client VM workload. In your sizing calculations, think of ESX as a fancy network switch. -- richard -- Richard Elling rich...@nexenta.com +1-760-896-4422 ZFS and NexentaStor training, Rotterdam, July 13-15, 2010 http://nexenta-rotterdam.eventbrite.com/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
