Mark thanks, This helped make things clearer for me also!
Tim "Mark D. Rodriguez" wrote: > Hi, > > I will attempt to clarify these issues here in regards to reclamation > and migration. My explanation is based on several years of experience, > course material that I have taught, and conversation with other ITSM > experts. Another words I have not seen the actual code and without that > no one can say for sure what it is doing. > > Migration: > > * Migration process(es) start with the himig threshold is passed. > The number of processes started is controlled by maxproc value. > * ITSM will evaluate which node has the most data in the storage > pool at that time. > * It then begins to move all of that nodes data on a per file space > basis, i.e. each file space will get its own process. > * If there are more processes available then there are file space > for the node with the most data then it will start to move the > node with the second most data, again a file space at a time. > * It will move all the nodes data before it re-evaluates its > threshold values. Therefore, once all the nodes data is moved it > looks at the threshold and if it needs to continue migrating then > it selects the node with the most data and start to move it. > * This continues until the low threshold is achieved. > * Therefore, it is possible for you to adjust the values and have > migration continue for sometime, even to the point of emptying the > pool beyond the low threshold value! Or it may stop right away > depending one the situation. > > Reclamation of Primary Pools (note copy pools are discussed below) > > * When a reclamation value is adjusted down to trigger reclamation a > list of tapes to be reclaimed is generated (visible in activity > log) and these are the tapes to be reclaimed. > * ITSM will start processing the tapes on the list. Each tape in > this reclamation will be a separate process. > * After completing a tape ITSM will re-evaluate the reclamation > threshold and decide whether to continue. > * Therefore, for primary storage pools you may in fact see the > reclamation process end without completing all the tapes on the > list, but it will not end (unless you cancel it) until the current > tape has finished reclaiming. > > Reclamation of Copy Pools > > * The process is similar for copy pools but it is different. The > big difference is that it process all the tapes at one time! > * When reclaiming copy pool tapes that are off site it uses a > primary copy of the data that is local. In order to improve the > reclamation process ITSM will mount the primary tape and move all > the NECESSARY files off that primary tape once it is mounted even > if it means it is in fact reclaiming files from 2 or more > different tapes. By doing this, there is no longer the concept of > reclaiming a single copy pool tape it is in fact reclaiming all > eligible tapes at once! > * Therefore, when you adjust the reclamation threshold upwards it > will not stop the reclamation of copy pool tapes. They only way > that process ends is for it to finish all the tapes or you cancel it. > > Again I will reiterate, I have not seen the code for any of these > processes, but I have through observation and through printed material > deduced this as being the methodology that ITSM is using. I am open to > others observations if they are different then mine. I wish that IBM > would just clearly state what the algorithm truly is. > > BTW, there are many techniques that have been talked about on this list > as ways of scripting around some of these difficulties. > > Good Luck and I hope this helped clear things up. > > -- > Regards, > Mark D. Rodriguez > President MDR Consulting, Inc. > > =============================================================================== > MDR Consulting > The very best in Technical Training and Consulting. > IBM Advanced Business Partner > SAIR Linux and GNU Authorized Center for Education > IBM Certified Advanced Technical Expert, CATE > AIX Support and Performance Tuning, RS6000 SP, TSM/ADSM and Linux > Red Hat Certified Engineer, RHCE > =============================================================================== > > Roger Deschner wrote: > > >That's not right. I just proved it, inadvertently. > > > >I wanted to deliberately empty out a Disk Storage Pool. I had set > >lowmig=0 highmig=0 migprocess=4 and it had dutifully mounted 4 tapes and > >started emptying things out in a hurry. > > > >Then 10:00 came, and a daily schedule I had set months ago to shut down > >migration at that time, happened, and set it to highmig=75 lowmig=25. > >(Classic self-inflicted foot-shooting.) The effect was to cancel all of > >the migration processes, even though the storage pool was still 10% > >full. These process cancelations took effect when each process reached > >the end of the current file, so they didn't all happen at once although > >it was fairly quick. Fortunately all the tapes were still mounted so I > >got them started again quickly. But what I proved (again) was that > >Migration will stop as soon as you change the thresholds, not when the > >original lowmig has been reached. > > > >If you are seeing something different, I wonder if your client nodes > >are backing up very large individual files? > > > >Reclamation is different from migration. While it does not work on a > >pre-built list, you are correct that it continues until the end of the > >volume it is currently reclaiming. Once a reclamation process finishes > >work on a volume, then the server examines the thresholds all over again > >to see if it wants to reclaim another tape. So if the reclamation > >threshold for that storage pool has been changed, it will take effect at > >that time - at the end of the volume currently being processed, not > >after all volumes in some list have been processed. As a result, if you > >are doing this on some schedule, and you need the tape drives at a > >particular time for something else such as database backup, you can > >either change the reclamation theshold several hours in advance, or > >develop an OS script to query the process, parse the result, and cancel > >it by process number. Even then, the reclamation process cancelation > >will not take effect until the end of the file currently being > >processed. > > > >The strategy I use to make sure I get the tape drives for database > >backup at the scheduled time, is to run migration between reclamation > >and DB backup, since migration can stop itself and free the drives > >pretty quickly, in contrast to reclamation. > > > >Roger Deschner University of Illinois at Chicago [EMAIL PROTECTED] > >======I have not lost my mind -- it is backed up on tape somewhere.===== > > > > > > > > > > > >>Date: Jan 04, 09:45 > >>From: Wheelock, Michael D <nobody at nowhere.com> > >> > >>Hi, > >> > >>When you start reclamation it begins a process and decides which tapes > >>it is going to reclaim based on the value given. These tapes can be > >>seen in the activity log of the server. It will continue to reclaim > >>until it has reclaimed all of the tapes in the list or you cancel the > >>reclamation process. =20 > >> > >>I have seen similar behavior in the migration process as well. If you > >>set a storage pool to migrate (ie. By setting high=3D0 and low=3D0) then = > >>it > >>will empty it out. If you subsequently reset the values (and the storage > >>pool is now above the low value and below the high value) it will > >>continue to migrate all of the data out. > >> > >>The migration_stop is there to set up the storage pools for the next > >>day's backup. The reclamation stop doesn't seem to be terribly useful. > >> > >>I set my storage pools to a decent value all of the time (something like > >>rec=3D90). Then on weekends (usually Friday afternoon) I set it to > >>something deeper (currently rec=3D75 especially for the storage pools on > >>the remote server). =20 > >> > >>Michael Wheelock > >>Integris Health > >> > >> > >> > >>-----Original Message----- > >>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On > >>Behalf Of > >>Tyree, David > >>Sent: Tuesday, January 04, 2005 8:18 AM > >>To: ADSM-L AT VM.MARIST DOT EDU > >>Subject: strange reclaimation behavor > >> > >> I have something strange going on with my reclaimations. I > >>have > >>the system issue a command in the afternoons to start reclaiming tapes > >>by > >>dropping the levels to 60%. I then have it issue another command around > >>midnight to raise the levels back to 100%. > >> > >> When I get here in the morning I will usally find that > >>reclaiming is still going on. I have verified that the stop_reclaim > >>scripts > >>are actually running by looking at the actlog for the time period > >>involved. > >>I end up doing a cancel process to get it to stop. > >> > >> I could understand it still running if it was in the middle > >>of a > >>50-60 gig file but it's only moving small (several meg) files. It's had > >>about eight hours to stop on it's own. I would have thought it would > >>have > >>found an opertunity to stop after eight hours. > >> > >> The contents of all of the scripts involved are correct. I > >>have > >>one starting the tapepool and another to start the copypool and another > >>couple to raise the level back up. I alternate different pools each > >>night. > >> > >> Any ideas here? > >> > >>I'm running 5.2.2 on a Win2k server with 4 LTO2 drives in a PV136T > >>library. > >> > >> > >> > >> > >> > >>David Tyree > >>Enterprise Backup Administrator > >>South Georgia Medical Center > >>229.333.1155 > >> > >>Confidential Notice: This e-mail message, including any attachments, is > >>for > >>the sole use of the intended recipient(s) and may contain confidential > >>and > >>privileged information. Any unauthorized review, use, disclosure or > >>distribution is prohibited. If you are not the intended recipient, > >>please > >>contact the sender by reply e-mail and destroy all copies of the > >>original > >>message. > >>*************************************************************************= > >>************************* > >>This e-mail may contain identifiable health information that is subject t= > >>o protection=20 > >>under state and federal law. This information is intended to be for the u= > >>se of the=20 > >>individual named above. If you are not the intended recipient, be aware t= > >>hat any=20 > >>disclosure, copying, distribution or use of the contents of this informat= > >>ion is prohibited=20 > >>and may be punishable by law. If you have received this electronic transm= > >>ission in=20 > >>error, please notify us immediately by electronic mail (reply). > >>*************************************************************************= > >>************************* > >> > >> > > > > > >
