Re: how not to run out of memory in cursor.execute

2006-06-10 Thread mjp
[EMAIL PROTECTED] wrote: > I am using cx_Oracle and MySQLdb to pull a lot of data from some tables > and I find that the cursor.execute method uses a lot of memory that > never gets garbage collected. Using fetchmany instead of fetchall does > not seem to make any difference, since it's the execute

Re: how not to run out of memory in cursor.execute

2006-06-07 Thread John Hunter
> "[EMAIL PROTECTED]" == [EMAIL PROTECTED] com <[EMAIL PROTECTED]> writes: [EMAIL PROTECTED]> whenever you are using a package that leaks memory. [EMAIL PROTECTED]> it can be appropriate to use Rpyc [EMAIL PROTECTED]> (http://rpyc.wikispaces.com/) to run the leaking [EMAIL PROT

Re: how not to run out of memory in cursor.execute

2006-06-07 Thread [EMAIL PROTECTED]
whenever you are using a package that leaks memory. it can be appropriate to use Rpyc (http://rpyc.wikispaces.com/) to run the leaking code in a different process, and restart it from time to time. I've been using this method to avoid the leaks of matplotlib. -- http://mail.python.org/mailman/lis

Re: how not to run out of memory in cursor.execute

2006-06-06 Thread John J. Lee
Jack Diederich <[EMAIL PROTECTED]> writes: [...] > MySQL will keep table locks until the results are all fetched so even though > the DB API allows fetchone() or fetchmany() using those with MySQLdb is > dangerous. [...] That's not true of InnoDB tables. John -- http://mail.python.org/mailman/l

Re: how not to run out of memory in cursor.execute

2006-06-06 Thread amberite
Steve Holden wrote: > > The MySQLdb solution you give is way more complicated than it needs to > be, thereby skewing your opinion towards cx_Oracle unnecessarily. > I respectfully disagree with your assertion here. The code I presented for MySQLdb is what you *have* to do, to avoid using up too

Re: how not to run out of memory in cursor.execute

2006-06-05 Thread Jack Diederich
On Mon, Jun 05, 2006 at 07:34:05PM +0100, Steve Holden wrote: > amberite wrote: > > [EMAIL PROTECTED] wrote: > > > >>I am using cx_Oracle and MySQLdb to pull a lot of data from some tables > >>and I find that the cursor.execute method uses a lot of memory that > >>never gets garbage collected. Usi

Re: how not to run out of memory in cursor.execute

2006-06-05 Thread Steve Holden
amberite wrote: > [EMAIL PROTECTED] wrote: > >>I am using cx_Oracle and MySQLdb to pull a lot of data from some tables >>and I find that the cursor.execute method uses a lot of memory that >>never gets garbage collected. Using fetchmany instead of fetchall does >>not seem to make any difference, s

Re: how not to run out of memory in cursor.execute

2006-06-05 Thread amberite
[EMAIL PROTECTED] wrote: > I am using cx_Oracle and MySQLdb to pull a lot of data from some tables > and I find that the cursor.execute method uses a lot of memory that > never gets garbage collected. Using fetchmany instead of fetchall does > not seem to make any difference, since it's the execut