Re: Python 3.2 bug? Reading the last line of a file

2011-05-27 Thread tkp...@hotmail.com
This is exactly what I want to do - I can then pick up various elements of the list and turn them into floats, ints, etc. I have not ever used decode, and will look it up in the docs to better understand it. I can't thank everyone enough for the generous serving of help and guidance - I certainly w

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread Jussi Piitulainen
tkp...@hotmail.com writes: > Looking through the docs did not clarify my understanding of the > issue. Why can I not split on '\t' when reading in binary mode? You can split on b'\t' to get a list of byteses, which you can then decode if you want them as strings. You can decode the bytes to get

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread Ian Kelly
On Wed, May 25, 2011 at 3:52 PM, MRAB wrote: > What do you mean by "may include the decoder state in its return value"? > > It does make sense that the values returned from tell() won't be in the > middle of an encoded sequence of bytes. If you take a look at the source code, tell() returns a lon

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread Ethan Furman
MRAB wrote: On 26/05/2011 00:25, tkp...@hotmail.com wrote: Thanks for the guidance - it was indeed an issue with reading in binary vs. text., and I do now succeed in reading the last line, except that I now seem unable to split it, as I demonstrate below. Here's what I get when I read the last l

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread MRAB
On 26/05/2011 00:25, tkp...@hotmail.com wrote: Thanks for the guidance - it was indeed an issue with reading in binary vs. text., and I do now succeed in reading the last line, except that I now seem unable to split it, as I demonstrate below. Here's what I get when I read the last line in text m

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread Ethan Furman
tkp...@hotmail.com wrote: Thanks for the guidance - it was indeed an issue with reading in binary vs. text., and I do now succeed in reading the last line, except that I now seem unable to split it, as I demonstrate below. Here's what I get when I read the last line in text mode using 2.7.1 and i

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread tkp...@hotmail.com
Thanks for the guidance - it was indeed an issue with reading in binary vs. text., and I do now succeed in reading the last line, except that I now seem unable to split it, as I demonstrate below. Here's what I get when I read the last line in text mode using 2.7.1 and in binary mode using 3.2 resp

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread MRAB
On 25/05/2011 21:54, Ian Kelly wrote: On Wed, May 25, 2011 at 2:00 PM, MRAB wrote: You're opening the file in text mode, and seeking relative to the end of the file is not allowed in text mode, presumably because the file contents have to be decoded, and, in general, seeking to an arbitrary pos

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread Ian Kelly
On Wed, May 25, 2011 at 2:00 PM, MRAB wrote: > You're opening the file in text mode, and seeking relative to the end > of the file is not allowed in text mode, presumably because the file > contents have to be decoded, and, in general, seeking to an arbitrary > position within a sequence of encode

Re: Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread MRAB
On 25/05/2011 20:33, tkp...@hotmail.com wrote: The following function that returns the last line of a file works perfectly well under Python 2.71. but fails reliably under Python 3.2. Is this a bug, or am I doing something wrong? Any help would be greatly appreciated. import os def lastLine(fi

Python 3.2 bug? Reading the last line of a file

2011-05-25 Thread tkp...@hotmail.com
The following function that returns the last line of a file works perfectly well under Python 2.71. but fails reliably under Python 3.2. Is this a bug, or am I doing something wrong? Any help would be greatly appreciated. import os def lastLine(filename): ''' Returns the last line of

Re: reading the last line of a file

2005-09-08 Thread Michael Sparks
Xah Lee wrote: > isn't there a way to implement tail in python with the same class of > performance? > > how's tail implemented?: Those crazy open source developers have an implementation here: http://cvs.sourceforge.net/viewcvs.py/mkcdrec/mkcdrec/busybox-0.60.5/Attic/tail.c?rev=1.1&view=markup I

Re: reading the last line of a file

2005-09-08 Thread Xah Lee
isn't there a way to implement tail in python with the same class of performance? how's tail implemented? Xah [EMAIL PROTECTED] ∑ http://xahlee.org/ Fredrik Lundh wrote: > Fredrik Lundh wrote: > > > zcat|tail is a LOT faster. > > and here's the "right way" to use that: > > from subprocess

Re: reading the last line of a file

2005-09-08 Thread Fredrik Lundh
Fredrik Lundh wrote: > zcat|tail is a LOT faster. and here's the "right way" to use that: from subprocess import Popen, PIPE p1 = Popen(["zcat", filename], stdout=PIPE) p2 = Popen(["tail", "-1"], stdin=p1.stdout, stdout=PIPE) last_line = p2.communicate()[0] (on my small sample,

Re: reading the last line of a file

2005-09-08 Thread Leif K-Brooks
Xah Lee wrote: > i switched to system call with tail because originally i was using a > pure Python solution > > inF = gzip.GzipFile(ff, 'rb'); > s=inF.readlines() > inF.close() > last_line=s[-1] > > and since the log file is 100 megabytes it takes a long time

Re: reading the last line of a file

2005-09-08 Thread Fredrik Lundh
Martin Franklin wrote: > Ok, in that case stick to your shell based solution, although 100 > megabytes does not sound that large to me I guess it is relative > to the system you are running on :) (I have over a gig of memory here) since the file is gzipped, you need to read all of it to get the l

Re: reading the last line of a file

2005-09-08 Thread Martin Franklin
Martin Franklin wrote: > Martin Franklin wrote: > >>Xah Lee wrote: >> >> >>>Martin Franklin wrote: >>> >>> >>> import gzip log_file = gzip.open("access_log.4.gz") last_line = log_file.readlines()[-1] log_file.close() >>> >>> >>>does the >>>log_file.readlines()[-1] >>>actually read

Re: reading the last line of a file

2005-09-08 Thread Martin Franklin
Martin Franklin wrote: > Xah Lee wrote: > >>Martin Franklin wrote: >> >> >>>import gzip >>>log_file = gzip.open("access_log.4.gz") >>>last_line = log_file.readlines()[-1] >>>log_file.close() >> >> >>does the >>log_file.readlines()[-1] >>actually read all the lines first? > > > > Yes I'm afraid

Re: reading the last line of a file

2005-09-08 Thread Martin Franklin
Xah Lee wrote: > Martin Franklin wrote: > >>import gzip >>log_file = gzip.open("access_log.4.gz") >>last_line = log_file.readlines()[-1] >>log_file.close() > > > does the > log_file.readlines()[-1] > actually read all the lines first? Yes I'm afraid it does. > > i switched to system call wi

reading the last line of a file

2005-09-08 Thread Xah Lee
Martin Franklin wrote: > import gzip > log_file = gzip.open("access_log.4.gz") > last_line = log_file.readlines()[-1] > log_file.close() does the log_file.readlines()[-1] actually read all the lines first? i switched to system call with tail because originally i was using a pure Python solution