Re: Reading from sys.stdin reads the whole file in

2014-08-28 Thread Akira Li
Chris Angelico writes: > On Wed, Aug 27, 2014 at 4:37 PM, Steven D'Aprano wrote: >> On Wed, 27 Aug 2014 08:29:20 +0300, Marko Rauhamaa wrote: >> >>> Try flushing after each print. >> >> Doesn't help. > > It does, but insufficiently. If slurp.py is run under Py3, it works > fine; or take Naoki's

Re: Reading from sys.stdin reads the whole file in

2014-08-27 Thread Peter Otten
Marko Rauhamaa wrote: > Peter Otten <__pete...@web.de>: > >> In addition to what already has been said: you can switch off output >> buffering of stdout/stderr with >> >> python -u out.py >> >> or by setting the PYTHONUNBUFFERED environment variable. > > Very often such externalities are not in

Re: Reading from sys.stdin reads the whole file in

2014-08-27 Thread Marko Rauhamaa
Peter Otten <__pete...@web.de>: > In addition to what already has been said: you can switch off output > buffering of stdout/stderr with > > python -u out.py > > or by setting the PYTHONUNBUFFERED environment variable. Very often such externalities are not in the control of the application develo

Re: Reading from sys.stdin reads the whole file in

2014-08-27 Thread Peter Otten
Steven D'Aprano wrote: > I'm trying to read from stdin. Here I simulate a process that slowly > outputs data to stdout: > > steve@runes:~$ cat out.py > import time > > print "Hello..." > time.sleep(10) > print "World!" > time.sleep(10) > print "Goodbye!" In addition to what already has been sai

Re: Reading from sys.stdin reads the whole file in

2014-08-27 Thread Steven D'Aprano
On Tue, 26 Aug 2014 23:07:36 -0700, Naoki INADA wrote: > for line in iter(sys.stdin.readline(), ''): Thanks for that. Removing the parens after readline seems to do the trick. -- Steven -- https://mail.python.org/mailman/listinfo/python-list

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Chris Angelico
On Wed, Aug 27, 2014 at 4:37 PM, Steven D'Aprano wrote: > On Wed, 27 Aug 2014 08:29:20 +0300, Marko Rauhamaa wrote: > >> Try flushing after each print. > > Doesn't help. It does, but insufficiently. If slurp.py is run under Py3, it works fine; or take Naoki's suggestion (although without the pare

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Steven D'Aprano
On Wed, 27 Aug 2014 08:29:20 +0300, Marko Rauhamaa wrote: > Steven D'Aprano : > >> When I pipe one to the other, I expect each line to be printed as they >> arrive, but instead they all queue up and happen at once: > > Try flushing after each print. Doesn't help. Here is an update that may mak

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Naoki INADA
I recommend Python 3. On Python 2, iterating lines without buffering is slow, tricky and ugly. for line in iter(sys.stdin.readline(), ''):     print line — Sent from Mailbox On Wed, Aug 27, 2014 at 3:03 PM, Chris Angelico wrote: > On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano wrote: >>

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Chris Angelico
On Wed, Aug 27, 2014 at 3:19 PM, Steven D'Aprano wrote: > When I pipe one to the other, I expect each line to be printed as they > arrive, but instead they all queue up and happen at once: You're seeing two different problems here. One is the flushing of stdout in out.py, as Marko mentioned, but

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Marko Rauhamaa
Marko Rauhamaa : > Try flushing after each print. http://stackoverflow.com/questions/230751/how-to-flush-ou tput-of-python-print> Since Python 3.3, there is no need to use sys.stdout.flush(): print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False) Marko -- https://mail.

Re: Reading from sys.stdin reads the whole file in

2014-08-26 Thread Marko Rauhamaa
Steven D'Aprano : > When I pipe one to the other, I expect each line to be printed as they > arrive, but instead they all queue up and happen at once: Try flushing after each print. When sys.stdout is a pipe, flushing happens only when the internal buffer fills up. Marko -- https://mail.pytho

Re: reading from sys.stdin

2007-04-15 Thread Diez B. Roggisch
>> Steve Holden +44 150 684 7255 +1 800 494 3119 >> Holden Web LLC/Ltd http://www.holdenweb.com >> Skype: holdenwebhttp://del.icio.us/steve.holden >> Recent Ramblings http://holdenweb.blogspot.com > > I just typed in 700 lines of text, and the iteration hasn't begun > yet.

Re: reading from sys.stdin

2007-04-15 Thread Gabriel Genellina
En Sun, 15 Apr 2007 16:51:12 -0300, 7stud <[EMAIL PROTECTED]> escribió: > I just typed in 700 lines of text, and the iteration hasn't begun > yet. Should I keep going? How much text each line? Python uses an 8K buffer. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-l

Re: reading from sys.stdin

2007-04-15 Thread 7stud
On Apr 15, 10:59 am, Steve Holden <[EMAIL PROTECTED]> wrote: > 7stud wrote: > > On Apr 14, 7:43 am, Steve Holden <[EMAIL PROTECTED]> wrote: > >> 7stud wrote: > >>> On Apr 13, 6:20 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: > >> [...] > > >>> But if you hit return on a blank line, there is no er

Re: reading from sys.stdin

2007-04-15 Thread Steve Holden
7stud wrote: > On Apr 14, 7:43 am, Steve Holden <[EMAIL PROTECTED]> wrote: >> 7stud wrote: >>> On Apr 13, 6:20 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: >> [...] >> >>> But if you hit return on a blank line, there is no error. In other >>> words, will stop on a blank line and not return EOFEr

Re: reading from sys.stdin

2007-04-14 Thread 7stud
On Apr 14, 7:43 am, Steve Holden <[EMAIL PROTECTED]> wrote: > 7stud wrote: > > On Apr 13, 6:20 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: > [...] > > > But if you hit return on a blank line, there is no error. In other > > words, will stop on a blank line and not return EOFError. > > > Anyway,

Re: reading from sys.stdin

2007-04-14 Thread Steve Holden
7stud wrote: > On Apr 13, 6:20 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: [...] > > But if you hit return on a blank line, there is no error. In other > words, will stop on a blank line and not return EOFError. > > Anyway, it seems everyone is saying that when you iterate over a file, > the

Re: reading from sys.stdin

2007-04-14 Thread Marc 'BlackJack' Rintsch
In <[EMAIL PROTECTED]>, 7stud wrote: > Anyway, it seems everyone is saying that when you iterate over a file, the > whole file is first read into memory. Therefore iterating over sys.stdin > is consistent: you have to type Ctrl+D to signal EOF before the iteration > can start. Is that about righ

Re: reading from sys.stdin

2007-04-14 Thread 7stud
On Apr 13, 6:20 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: > 7stud wrote: > > On Apr 13, 3:13 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: > >> 7stud wrote: > >>> I assume all input is buffered by default, so I'm not sure how it > >>> explains things to say that input from sys.stdin is buffer

Re: reading from sys.stdin

2007-04-13 Thread Michael Hoffman
7stud wrote: > On Apr 13, 3:13 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: >> 7stud wrote: >>> I assume all input is buffered by default, so I'm not sure how it >>> explains things to say that input from sys.stdin is buffered. >> The difference with sys.stdin is that it has indeterminate length

Re: reading from sys.stdin

2007-04-13 Thread Michael Bentley
On Apr 13, 2007, at 4:47 AM, 7stud wrote: > On Apr 13, 3:36 am, "7stud" <[EMAIL PROTECTED]> wrote: >> >>> It is if the file is smaller than the buffer size. >> >> How is that relevant? >> > > If I put 100 lines of text in a file with each line having 50 > characters, and I run this code: > > impo

Re: reading from sys.stdin

2007-04-13 Thread Marc 'BlackJack' Rintsch
In <[EMAIL PROTECTED]>, 7stud wrote: > On Apr 13, 3:36 am, "7stud" <[EMAIL PROTECTED]> wrote: >> >> > It is if the file is smaller than the buffer size. >> >> How is that relevant? >> > > If I put 100 lines of text in a file with each line having 50 > characters, and I run this code: > > import

Re: reading from sys.stdin

2007-04-13 Thread 7stud
On Apr 13, 3:36 am, "7stud" <[EMAIL PROTECTED]> wrote: > > > It is if the file is smaller than the buffer size. > > How is that relevant? > If I put 100 lines of text in a file with each line having 50 characters, and I run this code: import sys lst = [] for line in open("aaa.txt"): print "a

Re: reading from sys.stdin

2007-04-13 Thread 7stud
On Apr 13, 3:13 am, Michael Hoffman <[EMAIL PROTECTED]> wrote: > 7stud wrote: > > I assume all input is buffered by default, so I'm not sure how it > > explains things to say that input from sys.stdin is buffered. > > The difference with sys.stdin is that it has indeterminate length until > you sig

Re: reading from sys.stdin

2007-04-13 Thread Michael Hoffman
7stud wrote: > I assume all input is buffered by default, so I'm not sure how it > explains things to say that input from sys.stdin is buffered. The difference with sys.stdin is that it has indeterminate length until you signal EOF. I believe you'd get the same problem reading from, say, a name

Re: reading from sys.stdin

2007-04-13 Thread 7stud
Hi, Thanks for the responses. My book, Beginning Python: From Novice to Professional(p. 266) says that "sys.stdin is iterable, just like other files", so I thought I would test it out. However, I don't see how it is acting similar to a file in my example. I assume all input is buffered by defa

Re: reading from sys.stdin

2007-04-12 Thread Matimus
On Apr 12, 8:20 am, Maric Michaud <[EMAIL PROTECTED]> wrote: > Le jeudi 12 avril 2007 16:25, Matimus a écrit : > > > # Then you check to see if your file is interactive > > if f.isatty(): > > # This is unbuffered, and will iterate the same as f > > f = iter(raw_input(),"") > > This should b

Re: reading from sys.stdin

2007-04-12 Thread Maric Michaud
Le jeudi 12 avril 2007 16:25, Matimus a écrit : > # Then you check to see if your file is interactive > if f.isatty(): >     # This is unbuffered, and will iterate the same as f >     f = iter(raw_input(),"") This should be f = iter(raw_input,"") and this will end in a EOFError and stop on blank

Re: reading from sys.stdin

2007-04-12 Thread [EMAIL PROTECTED]
On Apr 12, 10:25 am, "Matimus" <[EMAIL PROTECTED]> wrote: > * well, not ALL, it will read in chunks. But, I think they are 4096 > Byte chunks by default. If you are referring to the read ahead buffer size, it is 8192 bytes. Raghu. -- http://mail.python.org/mailman/listinfo/python-list

Re: reading from sys.stdin

2007-04-12 Thread [EMAIL PROTECTED]
On Apr 12, 4:20 am, "7stud" <[EMAIL PROTECTED]> wrote: > I can't break out of the for loop in this example: > > -- > import sys > > lst = [] > for line in sys.stdin: > lst.append(line) > break > > print lst > --- You may want to look at a related issue: http://www.python.org/s

Re: reading from sys.stdin

2007-04-12 Thread Matimus
On Apr 12, 1:20 am, "7stud" <[EMAIL PROTECTED]> wrote: > I can't break out of the for loop in this example: > > -- > import sys > > lst = [] > for line in sys.stdin: > lst.append(line) > break > > print lst > --- > > But, I can break out of the for loop when I do this: > > -

Re: reading from sys.stdin

2007-04-12 Thread Steve Holden
Maric Michaud wrote: > Le jeudi 12 avril 2007 10:34, Diez B. Roggisch a écrit : >> I presume this is an OS thing. The first lines aren't communicated to >> the process until either the file is closed - C-d - or the buffer the OS >> puts before the stream is filled. You can switch to unbuffered behv

Re: reading from sys.stdin

2007-04-12 Thread Maric Michaud
Le jeudi 12 avril 2007 10:34, Diez B. Roggisch a écrit : > I presume this is an OS thing. The first lines aren't communicated to > the process until either the file is closed - C-d - or the buffer the OS > puts before the stream is filled. You can switch to unbuffered behviour > somehow, google for

Re: reading from sys.stdin

2007-04-12 Thread Gabriel Genellina
En Thu, 12 Apr 2007 05:20:58 -0300, 7stud <[EMAIL PROTECTED]> escribió: > I can't break out of the for loop in this example: > > -- > import sys > > lst = [] > for line in sys.stdin: > lst.append(line) > break > > print lst > --- Python 2.5.1c1 (r251c1:54692, Apr 5 2007, 09

Re: reading from sys.stdin

2007-04-12 Thread Michael Bentley
On Apr 12, 2007, at 3:20 AM, 7stud wrote: > I can't break out of the for loop in this example: > > -- > import sys > > lst = [] > for line in sys.stdin: > lst.append(line) > break > > print lst > --- > > But, I can break out of the for loop when I do this: > > - > impo

Re: reading from sys.stdin

2007-04-12 Thread Diez B. Roggisch
7stud schrieb: > I can't break out of the for loop in this example: > > -- > import sys > > lst = [] > for line in sys.stdin: > lst.append(line) > break > > print lst > --- Works for me. But only after the stdin is closed with a C-d. I presume this is an OS thing. The first