Python object overhead?

2007-03-23 Thread Matt Garman
I'm trying to use Python to work with large pipe ('|') delimited data files. The files range in size from 25 MB to 200 MB. Since each line corresponds to a record, what I'm trying to do is create an object from each record. However, it seems that doing this causes the memory overhead to go up tw

Re: Python object overhead?

2007-03-26 Thread Matt Garman
On 3/23/07, Bjoern Schliessmann <[EMAIL PROTECTED]> wrote: > "one blank line" == "EOF"? That's strange. Intended? In my case, I know my input data doesn't have any blank lines. However, I'm glad you (and others) clarified the issue, because I wasn't aware of the better methods for checking for EOF

Re: Python object overhead?

2007-03-26 Thread Matt Garman
On 3/23/07, Jack Diederich <[EMAIL PROTECTED]> wrote: > If you make the record a new style class (inherit from object) you can > specify the __slots__ attribute on the class. This eliminates the per > instance dictionary overhead in exchange for less flexibility. When you say "new style class", d