Re: Please help with MemoryError

2010-02-14 Thread Steve Howell
On Feb 11, 5:50 pm, Steven D'Aprano wrote: > On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: > > My Python program now consumes over 2 GB of memory and then I get a > > MemoryError.  I know I am reading lots of files into memory, but not 2GB > > worth. > > 2.    When do I need > > to manually a

Re: Please help with MemoryError

2010-02-14 Thread Steve Howell
On Feb 14, 10:32 am, Steve Holden wrote: > rantingrick wrote: > > On Feb 12, 4:10 pm, Steve Holden wrote: > >> Antoine Pitrou wrote: > >>> Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : > > > On Feb 12, 4:10 pm, Steve Holden wrote: > >> Antoine Pitrou wrote: > >>> Le Fri, 12 Feb 2

Re: Please help with MemoryError

2010-02-14 Thread Steve Holden
rantingrick wrote: > On Feb 12, 4:10 pm, Steve Holden wrote: >> Antoine Pitrou wrote: >>> Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : > > On Feb 12, 4:10 pm, Steve Holden wrote: >> Antoine Pitrou wrote: >>> Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : > > Stev

Re: Please help with MemoryError

2010-02-13 Thread rantingrick
On Feb 12, 4:10 pm, Steve Holden wrote: > Antoine Pitrou wrote: > > Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : On Feb 12, 4:10 pm, Steve Holden wrote: > Antoine Pitrou wrote: > > Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : Steve, Why do so many of your posts

Re: Please help with MemoryError

2010-02-12 Thread Gregory Ewing
Antoine Pitrou wrote: It's just that assignment ("=") means a different thing in Python than in non-object languages (or fake-object languages such as C++ or PHP): it rebinds instead of mutating in-place. If it mutated, you wouldn't have the AssertionError. It doesn't really have anything t

Re: Please help with MemoryError

2010-02-12 Thread Robert Kern
On 2010-02-12 17:30 PM, Antoine Pitrou wrote: Le Fri, 12 Feb 2010 23:49:38 +0100, Alf P. Steinbach a écrit : The main reason for not using that term for Python is that "pass by reference" has the extremely strong connotation of being able to implement 'swap'. But 'swap' is so easy to write as

Re: Please help with MemoryError

2010-02-12 Thread Gregory Ewing
Steven D'Aprano wrote: Python's calling convention already has an well-established name, established over thirty years ago by the distinguished computer scientist Barbara Liskov, namely call-by-sharing. And she was mistaken in thinking it needed a new name. -- Greg -- http://mail.python.org/

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 23:49:38 +0100, Alf P. Steinbach a écrit : > > The main reason for not using that term for Python is that "pass by > reference" has the extremely strong connotation of being able to > implement 'swap'. But 'swap' is so easy to write as a one-line statement that it's foolish t

Re: Please help with MemoryError

2010-02-12 Thread Steve Holden
Gib Bogle wrote: > Steven D'Aprano wrote: > >> def swap(a, b): >> a, b = b, a >> >> x = 1 >> y = 2 >> swap(x, y) >> assert (x == 2) and (y==1) > > Can't the same point be more simply made with this example: > > def setval(a): > a = 12345 > > x = 1 > setval(x) > print x > Yes, and it wi

Re: Please help with MemoryError

2010-02-12 Thread Gib Bogle
Steven D'Aprano wrote: def swap(a, b): a, b = b, a x = 1 y = 2 swap(x, y) assert (x == 2) and (y==1) Can't the same point be more simply made with this example: def setval(a): a = 12345 x = 1 setval(x) print x ? -- http://mail.python.org/mailman/listinfo/python-list

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Antoine Pitrou: Le Fri, 12 Feb 2010 23:12:06 +0100, Alf P. Steinbach a écrit : Steven talks about the standard meaning of "pass by reference". See my answer to Steve's message. You can't postulate a "standard meaning" of "pass by reference" independently of the specificities of each langua

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 23:12:06 +0100, Alf P. Steinbach a écrit : > > Steven talks about the standard meaning of "pass by reference". See my answer to Steve's message. You can't postulate a "standard meaning" of "pass by reference" independently of the specificities of each language. For example a

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 17:10:01 -0500, Steve Holden a écrit : > > As has already been pointed out, if Python used call by reference then > the following code would run without raising an AssertionError: > > def exchange(a, b): > a, b = b, a > > x = 1 > y = 2 > exchange(x, y) > assert (x == 2 an

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Antoine Pitrou: Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : What Python does is called "pass by sharing", or sometimes "pass by object reference". It is exactly the same as what (e.g.) Ruby and Java do, except that confusingly the Ruby people call it "pass by reference" and t

Re: Please help with MemoryError

2010-02-12 Thread Steve Holden
Antoine Pitrou wrote: > Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : >> What Python does is called "pass by sharing", or sometimes "pass by >> object reference". It is exactly the same as what (e.g.) Ruby and Java >> do, except that confusingly the Ruby people call it "pass by refe

Re: Please help with MemoryError

2010-02-12 Thread Antoine Pitrou
Le Fri, 12 Feb 2010 17:14:57 +, Steven D'Aprano a écrit : > > What Python does is called "pass by sharing", or sometimes "pass by > object reference". It is exactly the same as what (e.g.) Ruby and Java > do, except that confusingly the Ruby people call it "pass by reference" > and the Java pe

Re: Please help with MemoryError

2010-02-12 Thread Steven D'Aprano
On Fri, 12 Feb 2010 21:07:08 +0100, mk wrote: > John Posner wrote: > >>> http://effbot.org/zone/call-by-object.htm >>> http://en.wikipedia.org/wiki/Evaluation_strategy > >> [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html > > Hmm how about "call by label-value"? Pyth

Re: Please help with MemoryError

2010-02-12 Thread Ethan Furman
mk wrote: John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about "call by label-value"? That is, you change labels by assignment, but pass the v

Re: Please help with MemoryError

2010-02-12 Thread Alf P. Steinbach
* Christian Heimes: mk wrote: Hmm how about "call by label-value"? Or "call by guido"? How do you like "call like a dutch"? :] Just a note: it might be more clear to talk about "pass by XXX" than "call by XXX". Unless you're talking about something else than argument passing. The standard

Re: Please help with MemoryError

2010-02-12 Thread Christian Heimes
mk wrote: > Hmm how about "call by label-value"? Or "call by guido"? How do you like "call like a dutch"? :] -- http://mail.python.org/mailman/listinfo/python-list

Re: Please help with MemoryError

2010-02-12 Thread Mel
mk wrote: > John Posner wrote: > >>> http://effbot.org/zone/call-by-object.htm >>> http://en.wikipedia.org/wiki/Evaluation_strategy > >> [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html > > Hmm how about "call by label-value"? Nothing egregiously wrong with it.. mayb

Re: Please help with MemoryError

2010-02-12 Thread mk
John Posner wrote: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy [1] http://mail.python.org/pipermail/edu-sig/2008-May/008583.html Hmm how about "call by label-value"? That is, you change labels by assignment, but pass the value of the

Re: Please help with MemoryError

2010-02-12 Thread John Posner
On 2/12/2010 12:14 PM, Steven D'Aprano wrote: On Fri, 12 Feb 2010 06:45:31 -0800, Jeremy wrote: You also confirmed what I thought was true that all variables are passed "by reference" so I don't need to worry about the data being copied (unless I do that explicitly). No, but yes. No, variabl

Re: Please help with MemoryError

2010-02-12 Thread Steven D'Aprano
On Fri, 12 Feb 2010 06:45:31 -0800, Jeremy wrote: > You also confirmed what I thought was true that all variables are passed > "by reference" so I don't need to worry about the data being copied > (unless I do that explicitly). No, but yes. No, variables are not passed by reference, but yes, you

Re: Please help with MemoryError

2010-02-12 Thread Tim Chase
Aahz wrote: Not quite. One critical difference between dbm and dicts is the need to remember to "save" changes by setting the key's valud again. Could you give an example of this? I'm not sure I understand what you're saying. Well, you're more likely to hit this by wrapping dbm with shelve

Re: Please help with MemoryError

2010-02-12 Thread Jeremy
On Feb 11, 6:50 pm, Steven D'Aprano wrote: > On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: > > My Python program now consumes over 2 GB of memory and then I get a > > MemoryError.  I know I am reading lots of files into memory, but not 2GB > > worth. > > Are you sure? > > Keep in mind that Pyt

Re: Please help with MemoryError

2010-02-12 Thread Aahz
In article , Tim Chase wrote: >Aahz wrote: >> Tim Chase wrote: >>> >>> Just to add to the mix, I'd put the "anydbm" module on the gradient >>> between "using a file" and "using sqlite". It's a nice intermediate >>> step between rolling your own file formats for data on disk, and having >>> to

Re: Please help with MemoryError

2010-02-12 Thread Tim Chase
Aahz wrote: Tim Chase wrote: Just to add to the mix, I'd put the "anydbm" module on the gradient between "using a file" and "using sqlite". It's a nice intermediate step between rolling your own file formats for data on disk, and having to write SQL since access is entirely like you'd do with

Re: Please help with MemoryError

2010-02-11 Thread Aahz
In article , Tim Chase wrote: > >Just to add to the mix, I'd put the "anydbm" module on the gradient >between "using a file" and "using sqlite". It's a nice intermediate >step between rolling your own file formats for data on disk, and having >to write SQL since access is entirely like you'd do

Re: Please help with MemoryError

2010-02-11 Thread Tim Chase
Jonathan Gardner wrote: Don't use Python variables to store data long-term. Instead, setup a database or a file and use that. I'd first look at using a file, then using SQLite, and then a full-fledged database like PostgreSQL. Just to add to the mix, I'd put the "anydbm" module on the gradient

Re: Please help with MemoryError

2010-02-11 Thread Steven D'Aprano
On Thu, 11 Feb 2010 15:39:09 -0800, Jeremy wrote: > My Python program now consumes over 2 GB of memory and then I get a > MemoryError. I know I am reading lots of files into memory, but not 2GB > worth. Are you sure? Keep in mind that Python has a comparatively high overhead due to its object-

Re: Please help with MemoryError

2010-02-11 Thread Jonathan Gardner
On Feb 11, 3:39 pm, Jeremy wrote: > I have been using Python for several years now and have never run into > memory errors… > > until now. > Yes, Python does a good job of making memory errors the least of your worries as a programmer. Maybe it's doing too good of a job... > My Python program no

Re: Please help with MemoryError

2010-02-11 Thread Alf P. Steinbach
* Jeremy: I have been using Python for several years now and have never run into memory errors… until now. My Python program now consumes over 2 GB of memory and then I get a MemoryError. I know I am reading lots of files into memory, but not 2GB worth. I thought I didn't have to worry about

Re: Please help with MemoryError

2010-02-11 Thread Stephen Hansen
On Thu, Feb 11, 2010 at 3:39 PM, Jeremy wrote: > My Python program now consumes over 2 GB of memory and then I get a > MemoryError. I know I am reading lots of files into memory, but not > 2GB worth. I thought I didn't have to worry about memory allocation > in Python because of the garbage col