[issue10841] binary stdio

STINNER Victor added the comment: For the record, this change (always set stdout and stderr in binary mode on Windows) introduced (at least???) the following regressions: - #11272: "input() has trailing carriage return on windows", fixed in Python 3.2.1 - #11395: "print(s) fails on Windows w

[issue10841] binary stdio

Glenn Linderman added the comment: Thanks for your work on this Victor, and other commenters also. -- ___ Python tracker ___ ___ Pyth

[issue10841] binary stdio

STINNER Victor added the comment: Tests pass on Windows 7 buildbot, the two other XP buildbots have unrelated issues. I also tested on my XP VM: all tests pass. So I close this issue. -- resolution: -> fixed status: open -> closed ___ Python tracke

[issue10841] binary stdio

STINNER Victor added the comment: I commited io_binary_windows.patch + parser_translate_newline.patch as r87824. I fixed the patch on the parser to avoid leak on newtok if translate_newlines() fails. -- Added file: http://bugs.python.org/file20304/parser_translate_newline-2.patch ___

[issue10841] binary stdio

Guido van Rossum added the comment: On Thu, Jan 6, 2011 at 1:20 PM, STINNER Victor wrote: > I don't know the effect of _setmode(stderr, O_BINARY) on calls to > fputs(stderr, ...) in Py_FatalError(). On Windows it will write lines ending in LF only instead of CRLF. Most tools to read text file

[issue10841] binary stdio

Changes by Peter Kleiweg : -- nosy: -pebbe ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue10841] binary stdio

STINNER Victor added the comment: New patch setting O_BINARY mode using _setmode() (on Windows) to stdin, stdout, stderr, but also to all FileIO() objects. The patch on Py_Main() is still needed because Python may use sys.stderr when it is still a stdprinter object, before the final TextIOWra

[issue10841] binary stdio

Glenn Linderman added the comment: Victor, Thanks for your interest and patches. msg125530 points out the location of the code where _all_ fds could be O_BINARYed, when passed in to open. I think this would make all fds open in binary mode, per Guido's comment... he made exactly the comment

[issue10841] binary stdio

Guido van Rossum added the comment: Not sure why I've been added, but I agree that all fds should always be in binary mode in Python 3. -- ___ Python tracker ___ __

[issue10841] binary stdio

STINNER Victor added the comment: > Using my patch, the interpreter fails with a SyntaxError on \r: > attached patch translates newlines to avoid this problem. See also issue #4628. -- ___ Python tracker

[issue10841] binary stdio

Changes by Antoine Pitrou : -- nosy: +benjamin.peterson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue10841] binary stdio

STINNER Victor added the comment: Using my patch, the interpreter fails with a SyntaxError on \r: attached patch translates newlines to avoid this problem. -- Added file: http://bugs.python.org/file20291/parser_translate_newline.patch ___ Python tra

[issue10841] binary stdio

STINNER Victor added the comment: Attached stdio_binary.patch always set stdin, stdout and stderr in binary mode on Windows (and not only with -u command line flag). I added the following comment, is it correct? /* don't translate newlines */ -- keywords: +patch nosy: +haypo Adde

[issue10841] binary stdio

Glenn Linderman added the comment: Makes sense to me. Still should document the open file parameter when passed an fd, and either tell the user that it should be O_BINARY, or that it will be O_BINARYd for them, whichever technique is chosen. But having two newline techniques is bad, and if

[issue10841] binary stdio

Changes by Peter Kleiweg : -- nosy: +pebbe ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue10841] binary stdio

Antoine Pitrou added the comment: > So -u does do 2/3 of what my windows_binary() does :) Should I switch > my test case to use stderr to demonstrate that it doesn't help with > that? Well, judging by the history of this code, selectively putting -u in binary mode may be justified by the fact

[issue10841] binary stdio

Glenn Linderman added the comment: stderr is notable by its absence in the list of O_BINARY adjustments. So -u does do 2/3 of what my windows_binary() does :) Should I switch my test case to use stderr to demonstrate that it doesn't help with that? I vaguely remember that early versions of

[issue10841] binary stdio

Glenn Linderman added the comment: Don't find "initstdio" "stdio" in pythonrun.c. Has it moved? There are precious few references to stdin, stdout, stderr in that module, mostly for attaching the default encoding. -- ___ Python tracker

[issue10841] binary stdio

Antoine Pitrou added the comment: Ok, actually it boils down to the following code in Modules/main.c: if (Py_UnbufferedStdioFlag) { #if defined(MS_WINDOWS) || defined(__CYGWIN__) _setmode(fileno(stdin), O_BINARY); _setmode(fileno(stdout), O_BINARY); #endif ... which explain

[issue10841] binary stdio

Glenn Linderman added the comment: Found it. The file browser doesn't tell what line number it is, but in _io/Fileio.c function fileio_init, there is code like #ifdef O_BINARY flags |= O_BINARY; #endif #ifdef O_APPEND if (append) flags |= O_APPEND; #endif if (fd >= 0) {

[issue10841] binary stdio

Antoine Pitrou added the comment: > I suppose the FileIO in _io is next to look at, wherever it can be found. I don't get what you're looking for. FileIO is involved in both the buffered and unbuffered cases, so it shouldn't make a difference. In any case, please look into Modules/_io

[issue10841] binary stdio

Glenn Linderman added the comment: I suppose the FileIO in _io is next to look at, wherever it can be found. -- ___ Python tracker ___ __

[issue10841] binary stdio

Antoine Pitrou added the comment: > But there has to be special handling somewhere for opening std*, > because they are already open, unlike other files. That is no doubt > where the bug is. Can you point me at that code? See initstdio() in Python/pythonrun.c --

[issue10841] binary stdio

Glenn Linderman added the comment: I can read and understand C well enough, having coded in it for about 40 years now... but I left C for Perl and Perl for Python, I try not to code in C when I don't have to, these days, as the P languages are more productive, overall. But there has to be spe

[issue10841] binary stdio

Antoine Pitrou added the comment: > Is there an easy way for me to find the code for -u? "-u" just passes 0 for the "buffering" argument to open() when creating stdout and stderr. Otherwise "buffering" equals -1. You can find equivalent code for open() in Lib/_pyio.py (the actual code in is in

[issue10841] binary stdio

Glenn Linderman added the comment: Is there an easy way for me to find the code for -u? I haven't learned my way around the Python sources much, just peeked in modules that I've needed to fix or learn something from a little. I'm just surprised you think it is orthogonal, but I'm glad you a

[issue10841] binary stdio

Antoine Pitrou added the comment: > Actually, it seems like this "-u" behaviour, should simply be the > default for Python 3.x on Windows. There is nothing in "-u" which should explain the behaviour you're observing, so either way it's a bug. Making "-u" the default is quite orthogonal. --

[issue10841] binary stdio

Glenn Linderman added the comment: Actually, it seems like this "-u" behaviour, should simply be the default for Python 3.x on Windows. The new IO subsystem seems to be able to add \r when desired anyway. And except for Notepad, most programs on Windows can deal with \r\n or solo \n anyway.

[issue10841] binary stdio

Glenn Linderman added the comment: The same. This can be tested with the same test program, c:\python32\python.exe test.py 1 > test1.txt similar for 2, 3, 4. Then add -u and repeat. All 8 cases produce the same results, either via a pipe, or with a redirected stdout. -- _

[issue10841] binary stdio

Antoine Pitrou added the comment: What are the results if, instead of piping through subprocess, you simply redirect stdout to a file (using "... > myfile.txt")? -- ___ Python tracker

[issue10841] binary stdio

Changes by Antoine Pitrou : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue10841] binary stdio

Changes by Antoine Pitrou : -- components: +Windows nosy: +amaury.forgeotdarc, brian.curtin versions: +Python 3.1, Python 3.2 ___ Python tracker ___ _

[issue10841] binary stdio

Glenn Linderman added the comment: tested on Windows, for those that aren't following issue 4953 -- components: +IO type: -> behavior ___ Python tracker ___ ___

[issue10841] binary stdio

New submission from Glenn Linderman : Per Antoine's request, I wrote this test code, it isn't elegant, I whipped it together quickly; but it shows the issue. The issue may be one of my ignorance, but it does show the behavior I described in issue 4953. Here's the output from the various test