Hi, I have now tried different approaches for my large TSM query:
DBI: Allen Rout confirmed my suspicions in a previous post. This one loads the whole output of the sql into memory before processing. For my large query I killed the program when it reached 1GB. ODBC: This one seems to kick off the query and start processing immediately. However it still uses a lot of memory if not as much as DBI. My query topped out at around 400 MB. I am not sure what is going on under the hood here. Command pipe: open OUT, "dsmadmc -noconfirm -tabd -id=xxx -password=xxx select .... |"; This worked well, perl just using 4MB of memory. No memory problems on the TSM-server either. Thank you to all who responded! Best regards, Hans Chr. Riksheim -----Original Message----- From: ADSM: Dist Stor Manager [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Raibeck Sent: Monday, December 04, 2006 3:42 PM To: ADSM-L@VM.MARIST.EDU Subject: Re: Using the DBI::TSM module, memory problem I cannot remember if I ever tried using DBI, but I have been able to get Perl to work with the TSM ODBC driver by using the Roth ODBC package for Perl. Go to the URL in my sig, then do a search on: odbc perl to find an article on using Perl with the TSM ODBC driver. I believe my example in the article does not have the memory consumption issue that you describe. Regards, Andy Andy Raibeck IBM Software Group Tivoli Storage Manager Client Development Internal Notes e-mail: Andrew Raibeck/Tucson/[EMAIL PROTECTED] Internet e-mail: [EMAIL PROTECTED] IBM Tivoli Storage Manager support web page: http://www.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageManager.html The only dumb question is the one that goes unasked. The command line is your friend. "Good enough" is the enemy of excellence. "ADSM: Dist Stor Manager" <ADSM-L@VM.MARIST.EDU> wrote on 12/04/2006 06:21:07 AM: > Hi, > > anybody else using the perl module DBI::TSM? > > I've recently been using it for reporting purposes and find it quite > useful. However I have some problems with the perl memory consumption. > This may very well be caused by my lack of knowledge of DBI, but I > haven't found a remedy yet. > > The problem is that I want to process the output of sql statements > line by line as it is returned by the TSM-server, but it seems that > DBI requires the output of the whole sql to be stored in memory before > I can process it. For large queries the memory consumption will be > huge. > > Example(I know this is a simple one where it could be all handled in > the sql itself, this is just for explaining my problem): > > > # Set up the database handle: > $dbh=DBI->connect("DBI::TSM:myserver", 'admin', 'password¨'); > > # SQL that will return a zillion rows > $sql='select description as "DESC" from archives where node_name='MYNODE''; > > $sth=$dbh->prepare($sql); > > $sth->execute; #This is the one that slurps my memory > > while($row_ref)=$sth->fetchrow_hashref){ > #this processes each row that is already stored in memory > $archive_names{$row_ref->{'DESC'}}=''; > } > > > I guess I could avoid the problem by using "dsmadmc" directly and > write the output to a file that i could later have perl parse, but > that would not be a very elegant solution. I want it to work with DBI. > > > Any ideas? > > > Best regards, > > Hans Chr. Riksheim > > > > > > > <html><body><br /><br /><br /><font face="Arial" size="1"><hr>This > email originates from Steria AS, Biskop Gunnerus' gate 14a, N-0051 > OSLO, > http://www.steria.no. This email and any attachments may contain > confidential/intellectual property/copyright information and is only > for the use of the addressee(s). You are prohibited from copying, > forwarding, disclosing, saving or otherwise using it in any way if you > are not the addressee(s) or responsible for delivery. If you receive > this email by mistake, please advise the sender and cancel it > immediately. Steria may monitor the content of emails within its > network to ensure compliance with its policies and procedures. Any > email is susceptible to alteration and its integrity cannot be > assured. Steria shall not be liable if the message is altered, > modified, falsified, or even edited.</font><html><body><br /><br /><br > /><font face="Arial" size="1"><hr>This email originates from Steria AS, > Biskop Gunnerus' gate 14a, N-0051 OSLO, http://www.steria.no. This email and > any attachments may contain confidential/intellectual property/copyright > information and is only for the use of the addressee(s). You are prohibited > from copying, forwarding, disclosing, saving or otherwise using it in any way > if you are not the addressee(s) or responsible for delivery. If you receive > this email by mistake, please advise the sender and cancel it immediately. > Steria may monitor the content of emails within its network to ensure > compliance with its policies and procedures. Any email is susceptible to > alteration and its integrity cannot be assured. Steria shall not be liable if > the message is altered, modified, falsified, or even > edited.</font></body></html></body></html>
