Re: Sort Big File Help

2010-03-03 Thread Jonathan Gardner
On Wed, Mar 3, 2010 at 8:19 AM, John Filben wrote: > I am new to Python but have used many other (mostly dead) languages in the > past.  I want to be able to process *.txt and *.csv files.  I can now read > that and then change them as needed – mostly just take a column and do some > if-then to cr

Re: Sort Big File Help

2010-03-03 Thread mk
MRAB wrote: [snip] Simpler would be: lines = f.readlines() lines.sort(key=lambda line: line[ : 3]) or even: lines = sorted(f.readlines(), key=lambda line: line[ : 3])) Sure, but a complete newbie (I have this impression about OP) doesn't have to know about lambda. I expected

Re: Sort Big File Help

2010-03-03 Thread mk
John, there's an error in my program, I forgot that list.sort() method doesn't return the list (it sorts in place). So it should look like: #!/usr/bin/python def sortit(fname): fo = open(fname) linedict = {} for line in fo: key = line[:3] linedict[key] = line sor

Re: Sort Big File Help

2010-03-03 Thread Arnaud Delobelle
MRAB writes: > mk wrote: >> John Filben wrote: >>> I am new to Python but have used many other (mostly dead) languages >>> in the past. I want to be able to process *.txt and *.csv files. >>> I can now read that and then change them as needed – mostly just >>> take a column and do some if-then t

Re: Sort Big File Help

2010-03-03 Thread MRAB
mk wrote: John Filben wrote: I am new to Python but have used many other (mostly dead) languages in the past. I want to be able to process *.txt and *.csv files. I can now read that and then change them as needed – mostly just take a column and do some if-then to create a new variable. My p

Re: Sort Big File Help

2010-03-03 Thread mk
John Filben wrote: I am new to Python but have used many other (mostly dead) languages in the past. I want to be able to process *.txt and *.csv files. I can now read that and then change them as needed – mostly just take a column and do some if-then to create a new variable. My problem is s

Sort Big File Help

2010-03-03 Thread John Filben
I am new to Python but have used many other (mostly dead) languages in the past.  I want to be able to process *.txt and *.csv files.  I can now read that and then change them as needed – mostly just take a column and do some if-then to create a new variable.  My problem is sorting these files:

Re: Big file

2008-03-15 Thread hdante
On Mar 12, 10:50 pm, "Andrew Rekdal" <[EMAIL PROTECTED]> wrote: > Well, I can see how this could get real messy but within defining a GUI > there are many elements and so the block of elements such as a wx.notebook > for instance I would hope I could place all the code for this in another > file an

Re: Big file

2008-03-12 Thread Andrew Rekdal
Well, I can see how this could get real messy but within defining a GUI there are many elements and so the block of elements such as a wx.notebook for instance I would hope I could place all the code for this in another file and somehow include it into place. This way I can work on layered pane

Re: Big file

2008-03-12 Thread hdante
On Mar 12, 9:42 pm, "Andrew Rekdal" <[EMAIL PROTECTED]> wrote: > I am working in the class constructor defining elements of an application. > The problem is the file is getting unmanageble and I am wanting to extend the > contructor __init__ to another file. > > Is it possible to import directly

Re: Big file

2008-03-12 Thread Steven D'Aprano
On Wed, 12 Mar 2008 19:42:44 -0500, Andrew Rekdal wrote: > I am working in the class constructor defining elements of an > application. The problem is the file is getting unmanageble and I am > wanting to extend the contructor __init__ to another file. > > Is it possible to import directly into t

Re: Big file

2008-03-12 Thread Simon Forman
On Mar 12, 5:42 pm, "Andrew Rekdal" <[EMAIL PROTECTED]> wrote: > I am working in the class constructor defining elements of an application. > The problem is the file is getting unmanageble and I am wanting to extend the > contructor __init__ to another file. > > Is it possible to import directly

Big file

2008-03-12 Thread Andrew Rekdal
I am working in the class constructor defining elements of an application. The problem is the file is getting unmanageble and I am wanting to extend the contructor __init__ to another file. Is it possible to import directly into the contructor the contents of another module file? If so how wou

Re: looping over a big file

2005-07-05 Thread Asun Friere
sorry lost the first line in pasting: Python 2.4.1 (#1, Jun 21 2005, 12:38:55) :/ -- http://mail.python.org/mailman/listinfo/python-list

Re: looping over a big file

2005-07-05 Thread Asun Friere
Jp Calderone wrote: > fileIter = iter(big_file) > for line in fileIter: > line_after = fileIter.next() > > Don't mix iterating with any other file methods, since it will confuse the > buffering scheme. > Isn't a file an iterable already? [GCC 3.3.3 20040412 (Red Hat Linux 3.3.3-7)]

Re: looping over a big file

2005-07-04 Thread Terry Hancock
t; "always with a file"). WHEN it returns things other than files. Like a StringIO object, which can be quite handy. True, it won't be a "big file", but it'd be nice if the same code would tolerate it. I've used this with e.g. PIL quite a bit when working with

Re: looping over a big file

2005-07-03 Thread Peter Hansen
Michael Hoffman wrote: > Mike Meyer wrote: >> Guido has made a pronouncement on open vs. file. I think he prefers >> open for opening files, and file for type testing, but may well be >> wrong. I don't think it's critical. > > He has said that open() may be used for things other than files in the

Re: looping over a big file

2005-07-03 Thread Michael Hoffman
Mike Meyer wrote: > Roy Smith <[EMAIL PROTECTED]> writes: > >>The "right" way to do this is: >> >>for line in file ("filename"): >> whatever >> >>The file object returned by file() acts as an iterator. Each time through >>the loop, another line is read and returned (I'm sure there is some >>b

Re: looping over a big file

2005-07-03 Thread Mike Meyer
Roy Smith <[EMAIL PROTECTED]> writes: > The "right" way to do this is: > > for line in file ("filename"): >whatever > > The file object returned by file() acts as an iterator. Each time through > the loop, another line is read and returned (I'm sure there is some > block-level buffering goin

Re: looping over a big file

2005-07-03 Thread Roy Smith
Jp Calderone <[EMAIL PROTECTED]> wrote: > Yes, but you need to do it like this: > > fileIter = iter(big_file) > for line in fileIter: > line_after = fileIter.next() That's better than the solution I posted. -- http://mail.python.org/mailman/listinfo/python-list

Re: looping over a big file

2005-07-03 Thread Jp Calderone
On Sun, 3 Jul 2005 23:52:12 +0200, martian <[EMAIL PROTECTED]> wrote: >Hi, > >I've a couple of questions regarding the processing of a big text file >(16MB). > >1) how does python handle: > >> for line in big_file: > >is big_file all read into memory or one line is read at a time or a buffer >is us

Re: looping over a big file

2005-07-03 Thread Roy Smith
"martian" <[EMAIL PROTECTED]> wrote: > 1) how does python handle: > > > for line in big_file: > > is big_file all read into memory or one line is read at a time or a buffer > is used or ...? The "right" way to do this is: for line in file ("filename"): whatever The file object returned by f

looping over a big file

2005-07-03 Thread martian
Hi, I've a couple of questions regarding the processing of a big text file (16MB). 1) how does python handle: > for line in big_file: is big_file all read into memory or one line is read at a time or a buffer is used or ...? 2) is it possible to advance lines within the loop? The following doe