We were lucky to afford the pair of 3TB SSD on the target server, which is dedicated to the DB.
Since our primary disk these days is ISILON/NFS, I am trying to stuff as much rotating disk into the servers as they are replaced. The one we are about to switch to has 12-10TB internal disk (for LZ/primary storage - not including OS, actlog, archlog), 72-threads, 256GB RAM and quad 10G. I figure the 100TB internal storage will hold 1/3 of this servers onsite occupancy, allowing me to dedup and start getting away from tape for onsite storage. On Fri, Apr 27, 2018 at 3:58 AM, Stefan Folkerts <stefan.folke...@gmail.com> wrote: > We have just build a setup at a large university that replicates (on > average) 6.9TB per hour but that data is deduplicated on the source server. > If you have 10Gb/s+ bandwith and no limitation on the performance on the > source you should but able to handle 7TB per day (mixed workload, not only > tiny files) on a target server if it's an M blueprint or faster model > without any issue. > The specifications of the compute hardware at this customer are less then > what you specify memory wise, we are spot on M blueprint but with NVME's > for the database. > If your source server can deliver the data fast enough it's all about super > fast database and activelog performance on the target, it's needs to do an > insane amount of iop/s to chunk, hash and check all the chunks you are > throwing at it. And yes, memory helps but 256GB isn't as important as > database speed and raw CPU power to plow thru that data. > > Did you run benchmarks on the database and activelog volumes on your target > server? > We reach about 110.000 IOP/s on the database volumes using NVME and we have > found that to be the key to unlocking Spectrum Protect ludicrous-mode. > > https://imgur.com/a/SAA7OAZ > > > On Thu, Apr 26, 2018 at 10:37 PM, Skylar Thompson <skyl...@uw.edu> wrote: > > > Are you CPU or disk-bound on the source or target servers? Even if you > have > > lots of CPUs, replication might be running on a single thread and just > > using > > one CPU. > > > > On Thu, Apr 26, 2018 at 02:46:24PM -0400, Zoltan Forray wrote: > > > As we get deeper into Replication and my boss wants to use it more and > > more > > > as an offsite recovery platform. > > > > > > As we try to reach "best practices" of replicating everything, we are > > > finding this desire to be difficult if not impossible to achieve due to > > the > > > resource demands. > > > > > > Total we want to eventually replicate is around 700TB from 5-source > > servers > > > to 1-target server which is dedicated to replication. > > > > > > So the big question is, can this be done? > > > > > > We recently rebuilt the offsite target server to as big as we could > > afford > > > ($38K). It has 256GB of RAM. 64-threads of CPU. Storage is primarily > > > 500TB of ISILON/NFS. Connectivity is via quad 10G (2-for IP traffic > from > > > source servers and 2-for ISILON/NFS). > > > > > > Yet we can only replicate around 3TB daily when we backup around 7TB. > > > > > > Looking for suggestions/thoughts/experiences? > > > > > > All boxes are RHEL Linux and 7.1.7.300 > > > > > > -- > > > *Zoltan Forray* > > > Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator > > > Xymon Monitor Administrator > > > VMware Administrator > > > Virginia Commonwealth University > > > UCC/Office of Technology Services > > > www.ucc.vcu.edu > > > zfor...@vcu.edu - 804-828-4807 > > > Don't be a phishing victim - VCU and other reputable organizations will > > > never use email to request that you reply with your password, social > > > security number or confidential personal information. For more details > > > visit http://phishing.vcu.edu/ > > > > -- > > -- Skylar Thompson (skyl...@u.washington.edu) > > -- Genome Sciences Department, System Administrator > > -- Foege Building S046, (206)-685-7354 > > -- University of Washington School of Medicine > > > -- *Zoltan Forray* Spectrum Protect (p.k.a. TSM) Software & Hardware Administrator Xymon Monitor Administrator VMware Administrator Virginia Commonwealth University UCC/Office of Technology Services www.ucc.vcu.edu zfor...@vcu.edu - 804-828-4807 Don't be a phishing victim - VCU and other reputable organizations will never use email to request that you reply with your password, social security number or confidential personal information. For more details visit http://phishing.vcu.edu/
