New submission from Yann Dirson <y...@blade-group.com>:

On a POpen object created with bufsize=0, stdout.readline() does a buffered 
reading with python3, whereas in 2.7 it did char-by-char reading.  See attached 
example.

As a result, a poll including the stdout object suffers a behaviour change when 
stdout is ready for writing and there is more than one line of data available.  
In both cases we get notified by poll() that data is available on the fd and we 
can stdout.readline() and get back to our polling loop.  Then:

* with python2 poll() then returns immediately and stdout.readline() will then 
return the next line

* with python3 poll() now blocks

Running the attached example under strace reveals the underlying difference:

 write(4, "go\n", 3)                     = 3
 poll([{fd=5, events=POLLIN|POLLERR|POLLHUP}], 1, -1) = 1 ([{fd=5, 
revents=POLLIN}])
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "x", 1)                         = 1
-read(5, "\n", 1)                        = 1
-fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0x2), ...}) = 0
+read(5, "xxxxxxxxxxxx\nyyyyyyyyyyyyyyy\naaa"..., 8192) = 74
 write(1, ">xxxxxxxxxxxx\n", 14)         = 14


We can see a buffered read, which explains the behaviour difference.

Changing to bufsize=1, strace does not show a difference here.

This is especially troubling, as the first note in 
https://docs.python.org/3/library/io.html#class-hierarchy mentions that even in 
buffered mode there is an unoptimized readline() implementation.

----------
components: IO
files: testproc-unbuffered.py
messages: 373165
nosy: yann
priority: normal
severity: normal
status: open
title: Undocumented behaviour change of POpen.stdout.readine with bufsize=0 or 
=1
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file49299/testproc-unbuffered.py

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue41222>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to