Any further improvements would probably have to be hardware related. The bottleneck with groupwise backups is usually the file-scan speed of the bus/controller/disks/raidset.
>>> [EMAIL PROTECTED] 9/13/2006 12:53 PM >>> ---------------------------- Top of message ---------------------------- >>--> 09-13-06 10:49 S.SHEPPARD (SHS) Re: Groupwise Backup We set 'enablecaching' to no which seemed to have no effect. It appears from what my Netware people are telling me, that the 'enablegroupwise' switch has a setting of 1 or 0; don't see a 'true' option. Can someone explain to me what this might do? At this point, after several different tweaks we have gotten the throughput up to around 30GB/hour, and think we will probably be able to live with that, unless anyone has any other suggestions. Thanks Sam Sheppard San Diego Data Processing Corp. (858)-581-9668 -----------------------------------------------------------------------` ---------------------------- Top of message ---------------------------- >>--> 09-13-06 02:10 ..NETMAIL (001) Re: [ADSM-L] Groupwise Ba Date: Wed, 13 Sep 2006 19:02:53 +1000 From: "Marco Malgarini" <[EMAIL PROTECTED]> Organization: Malga Consulting Subject: Re: [ADSM-L] Groupwise Backup To: ADSM-L@VM.MARIST.EDU _________________________________Top_of_Message_________________________________ Hi Troy TSM supported that switch with client 5.3.0.12 and we get fault free backups and 100% quality restores on 100GB and larger post offices (we are backing up 100+ of these post offices state-wide, lots of test experience). We are currently testing the 5.3.4.0 client in this configuration and didn't have any errors yet. Server version 5.3.3.2 on W2K3 server Kind Regards Marco Malgarini Malga Consulting -----Original Message----- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Troy Frank Sent: Wednesday, 13 September 2006 08:10 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Groupwise Backup TSM does not support the use of the /enableGroupwise switch, as it is the replacement for tsagw, which tsm also didn't support. It doesn't stop backups from running, but it caused more errors in my experience. The /nocachingmode switch is important to do though...had forgotten about that one. You can also set it permanently by editing the sys:/etc/sms/tsa.cfg file, and changing the "Enable Caching" option from "yes", to "no". >>> Marco Malgarini <[EMAIL PROTECTED]> 9/12/2006 4:49 PM >>> One more suggestion: Have you tried the following TSAFS settings? Tsafs /enableGroupwise=true this is a TSAFS load switch And have you tried tsafs /nocachingmode this is an online switch which we use as a pre-schedule command. Our post offices have about 1,500,000 objects and inspection time alone takes between 30min to 1 1/2 hours depending of other backups running on the same box. i.e. file and print data. Kind Regards Marco Malgarini Malga Consulting -----Original Message----- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Sam Sheppard Sent: Wednesday, 13 September 2006 06:58 AM To: ADSM-L@VM.MARIST.EDU Subject: Re: [ADSM-L] Groupwise Backup ---------------------------- Top of message ---------------------------- >>--> 09-12-06 13:43 S.SHEPPARD (SHS) Re: Groupwise Backup I'll try scaling back the TXNG, but the data is static (snapshot copy), so nothing is changing. My understanding of RESOURCEUTILIZATION is that you need multiple volumes (filespaces) to use multiple producer threads and since we're only backing up the one Groupwise volume I don't think RESOURCEUTILIZATION will buy us anything. I'll also try upping the TCPBUFFSIZE. I was taking the recommendation from the Performance Tuning Guide, but I have plenty of memory to play with. Thanks Sam Sheppard San Diego Data Processing Corp. (858)-581-9668 -----------------------------------------------------------------------` ---------------------------- Top of message ---------------------------- >>--> 09-12-06 13:38 ..NETMAIL () Re: [ADSM-L] Groupwise Ba Date: Tue, 12 Sep 2006 15:32:11 -0500 From: "Troy Frank" <[EMAIL PROTECTED]> Subject: Re: [ADSM-L] Groupwise Backup To: ADSM-L@VM.MARIST.EDU _________________________________Top_of_Message_____________________________ ____ Couple suggestions... Server side - groupwise tends to work better with a smaller TXNGROUPMAX (ours is 1024). Since groupwise is a lot of small files, and they can change rapidly, big TXNGROUPMAX settings can equal a lot of aggregate rebuilding before going to the server. Client Side - Set RESOURCEUTILIZATION to 4 or so. Netware seems to have issues with going higher than 5 or 6, but 4 seems to work well. You'll get more filesystem reader/data sender threads. I've included some of our client-side dsm.opt settings below... ResourceUtilization 4 TCPBUFFSIZE 127 TCPWINDOWSIZE 64 TXNBYTELIMIT 25600 LARGECOMmbuffers Yes I've read some things that suggest LARGECOMmbuffers on netware is a completely useless command, but it doesn't seem to hurt anything, so I haven't taken it out. Also keep in mind that groupwise backups will always be slower than most other types of systems. It's a pretty worst-case scenario for an incremental backup since it involves a huge number of very small files. An ftp transfer doesn't really accurately tell you anything, since that's only testing speed of reading a single/few very large files, and filling the network pipe. Disk performance will likely be the dominating factor in this kind of scenario, so I'd look at what kind of raid set you have, how many spindles, on how many controllers, with what block size, what rpm your drives are, ect. >>> [EMAIL PROTECTED] 9/12/2006 1:45 PM >>> We have installed a new TSM server intended to backup about a dozen Novell Groupwise post offices, totaling around 600GB. These are being directed to IBM Ultrium-TD3 LTO tapes in a FC-attached Spectralogic library. The TSM server is Version 5.3.3.3 running under Solaris 10 (Sunfire V240, 2 1.5G CPUs, 8GB memory). The clients machines are dual-processor 3GHz, 3GB memory running the 5.3.4 version of the Netware client. Client/Server connection is over a GigE VLan. Server options: COMMmethod TCPIP TCPWindowsize 128 BUFPOOLSIZE 128000 EXPINTERVAL 0 SELFTUNEBUFPOOLSIZE YES TXNGROUPMAX 2048 Client Options: COMMMETHOD TCPip TCPSERVERADDRESS 172.18.16.6 TCPBUFFSIZE 32 TCPWINDOWSIZE 64 TCPPORT 1500 TXNB 2097152 PASSWORDACCESS GENERATE PROCESSORUTILIZATION 100 MEMORYEFFICIENTBACKUP NO Initial test backup of a small (9GB) PO showed a throughput of around 20GB/hour. Subsequent tests have improved to 25-29GB/hour after upping the TXNG and PROCESSORUTILIZATION parms, but this still seems awfully slow for what, to us, seems like a pretty beefy system. The data resides on a NetApp FAS device and we actually backup a snapshot of the PO to avoid having to take Groupwise down. For reasons I won't go into, NDMP was removed as an option when putting this configuration together. Stats show 60%+ Comm. Wait, so I'm assuming this is a client-side or network issue, but I'm at a loss as to what to try next. FTP tests from the client show excellent throughput (200+GB/hour), so I don't believe it's a network issue. We're going to up the client-side TCPW to 128, but I'm not optomistic. Can anyone else out there give me their experiences, performance-wise, with large Groupwise backups and any hints at how to increase this throughput? I'm beginning to think it may be limitations of the Netware OS/client. We're hoping to get near 40GB/hour to make our window. Thanks in advance, Sam Sheppard San Diego Data Processing Corp. (858)-581-9668 Confidentiality Notice follows: The information in this message (and the documents attached to it, if any) is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken, or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this message in error, please delete all electronic copies of this message (and the documents attached to it, if any), destroy any hard copies you may have created and notify me immediately by replying to this email. Thank you. -----------------------------------------------------------------------` Confidentiality Notice follows: The information in this message (and the documents attached to it, if any) is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken, or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this message in error, please delete all electronic copies of this message (and the documents attached to it, if any), destroy any hard copies you may have created and notify me immediately by replying to this email. Thank you. -----------------------------------------------------------------------` Confidentiality Notice follows: The information in this message (and the documents attached to it, if any) is confidential and may be legally privileged. It is intended solely for the addressee. Access to this message by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken, or omitted to be taken in reliance on it is prohibited and may be unlawful. If you have received this message in error, please delete all electronic copies of this message (and the documents attached to it, if any), destroy any hard copies you may have created and notify me immediately by replying to this email. Thank you.
