[issue11395] print(s) fails on Windows with long strings

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue11395] print(s) fails on Windows with long strings

Eryk Sun added the comment: > the problem (or at least with these numbers) occurs only when > the code is saved in a script, and this is run by double- > clicking the file The .py file association is probably using a different version of Python, or it's associated with the py launcher and th

[issue11395] print(s) fails on Windows with long strings

Adam Bartoš added the comment: I've been hit by this issue recently. On my configuration, print("a" * 10215) fails with an infinite loop of OSErrors (WinError 8). This even cannot by interrupted with Ctrl-C nor the exception can be catched. - print("a" * 10214) is fine - print("a" * 10215) is

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: This issue is closed. You should reopen it or open a new one. -- ___ Python tracker ___ ___ Python-b

[issue11395] print(s) fails on Windows with long strings

eryksun added the comment: The buffer size only needs to be capped if WINVER < 0x602. This issue doesn't apply to Windows 8 since it uses the ConDrv device driver instead of LPC. Prior to Windows 8, WriteFile redirects to WriteConsoleA when passed a console handle. This makes an LPC call to c

[issue11395] print(s) fails on Windows with long strings

David-Sarah Hopwood added the comment: If I understand the bug in the Windows console functions correctly, a limit of 32767 bytes might not always be small enough. The problem is that if two or more threads are concurrently using any console functions (which all use the same 64 KiB heap), the

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: I realized that it was a little more difficult to port the fix on 3.1 because 3.1 doesn't have the fix for Windows 64 bits. So I only fixed Python 3.2 and 3.3, also because nobody reported failure for Python 3.1 on Windows with -u flag. I tested my fix: the

[issue11395] print(s) fails on Windows with long strings

Roundup Robot added the comment: New changeset 8939a21bdb94 by Victor Stinner in branch '3.2': Issue #11395: io.FileIO().write() clamps the data length to 32,767 bytes on http://hg.python.org/cpython/rev/8939a21bdb94 New changeset 4b3472169493 by Victor Stinner in branch 'default': (merge) Issu

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: > This last patch looks good, except that the comments "if stdout mode > is binary (python -u)" are incorrect: since r87824, all files are > opened in binary mode. I plan to commit the patch to 3.1 and then forward port to 3.2 and 3.3. Yes, I will adapt the

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: This last patch looks good, except that the comments "if stdout mode is binary (python -u)" are incorrect: since r87824, all files are opened in binary mode. -- ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: I tried to commit io_write.patch, but I had problems with Mercurial :-) I will commit it later. -- Added file: http://bugs.python.org/file21030/io_write.patch ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: Attached a version of the last patch without `.isatty` caching. -- Added file: http://bugs.python.org/file21025/winconsole_large_py33_direct.patch ___ Python tracker _

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: FWIW, here's the Microsoft's source for isatty (in VC\crt\src\isatty.c): /*** *int _isatty(handle) - check if handle is a device * *Purpose: * Checks if the given handle is associated with a character device * (terminal, console, printer, serial por

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: On Windows, isatty() is a cheap call: a simple lookup in the _ioinfo structure. And dup2() can still change the destination of a file descriptor, so the new attribute can be out of sync... I suggest to call isatty() on every write. --

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: Attached a modified patch that should work against 3.2+ heads: - Added `isatty` bit field in isatty that's evaluated during its construction. This should eliminate the need to call `isatty()` on every write. - Cap buffer length to 32767 (4 * 1024 - 1) when

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: Thanks for the comment. It's my first patch. :-) > - the patch doesn't apply on Python 3.3 That latest patch file I generated against the tip of 3.1 branch. Should I create two separate patches for 3.1 and 3.2+ (which will apply on 3.3, as well)? Actually, t

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: Other remarks about test_wconsole_binlarge.patch: - the patch doesn't apply on Python 3.3 - I would prefer 32767 instead of 32000 for the maximum length Suggestion for the comment in fileio.c: * Issue #11395: not enough space error (errno 12) on writing in

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: Remarks about test_wconsole_binlarge.patch: - I don't know if isatty() is cheap or not. Is it a system call? If it might be slow, it should be only be called once in the constructor. On Windows, I don't think that isatty(fd) evoles. - I don't want to commit

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: I did some tests: os.write(1, b'X'*length) does always fail with length >= 63842. It does sometimes fail with length > 35000. The maximum looks completly random: as written in Microsoft documentation, "The maximum size of the buffer will depend on heap usage"

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: > Indeed, Python3.1 fails with the -u option. I'm also attaching another test to reproduce the crash with '-u' option. -- Added file: http://bugs.python.org/file21013/test_wconsole_binlarge.patch ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: And a patch for the test + fix. -- Added file: http://bugs.python.org/file21012/wconsole_large.patch ___ Python tracker ___ ___

[issue11395] print(s) fails on Windows with long strings

Santoso Wijaya added the comment: I'm adding a test that will reproduce the crash. -- keywords: +patch Added file: http://bugs.python.org/file21003/writeconsole.patch ___ Python tracker ___

[issue11395] print(s) fails on Windows with long strings

Terry J. Reedy added the comment: print("a"*66000) works (after some delay) running from an IDLE edit window (but see #144249). Works means that I get a working prompt back with no errors. Unlike IDLE, the Command Prompt Windows keeps a limited number of lines in its buffers (default: 4x50),

[issue11395] print(s) fails on Windows with long strings

Changes by Santoso Wijaya : -- components: +Windows ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: It may be a windows bug, but it's also an a python regression! A fix is to limit the number of chars: === --- D:/py3k/Modules/_io/fileio.c (revision 87824) +++ D:/py3k/Modules/_io/fileio.c

[issue11395] print(s) fails on Windows with long strings

Antoine Pitrou added the comment: IIUC, this is a Windows bug? Is there any easy workaround for us? -- versions: +Python 3.3 ___ Python tracker ___ _

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: > And WriteConsole has the same issue. print() (sys.stdout and sys.stderr) should use WriteConsoleW() and use small chunks (smaller than 64 KB, I don't know the safest size). -- ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: I'm writing bytes here: os.write(1, b"b" * 66000) And WriteConsole has the same issue. -- ___ Python tracker ___

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: Anyway, use os.write() to write unicode into the Windows console is not the right thing to do. We should use WriteConsoleW(): #1602 is the correct fix for this issue. -- ___ Python tracker

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: Indeed, Python3.1 fails with the -u option. Before r87824, the C call to write() performed CRLF conversion. In the implementation of MSVCRT, a local buffer is used (1025 chars in vs8.0, 5*1024 in vs10.0), so WriteFile is called with small sizes. Since

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: > This changed with r87824 Yes, I changed Python to open all files in binary mode. With Python < 3.2, you can open sys.std* streams in binary mode using -u command line option (u like unbuffered, not Unicode ;-)). --

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: This changed with r87824 -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: Extract of the WriteConsole Function: << The storage for this buffer is allocated from a shared heap for the process that is 64 KB in size. The maximum size of the buffer will depend on heap usage. >> http://msdn.microsoft.com/en-us/library/ms687401(VS.85).as

[issue11395] print(s) fails on Windows with long strings

STINNER Victor added the comment: Extract of issue #1602: << WriteConsoleW has one bug that I know of, which is that it http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1232";>fails when writing more than 26608 characters at once. That's easy to work around by limiting the amount of data passed in

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: no, it works with 3.2b2 (r32b2:87398), and fails with 3.2 final (r32:88445) -- ___ Python tracker ___ ___

[issue11395] print(s) fails on Windows with long strings

Charles-Francois Natali added the comment: It's probably a Windows limitation regarding the number of bytes that can be written to stdout in one write. As for the difference between python versions, what does python -c "import sys; print(sys.getsizeof('a'))" return ? -- nosy: +neologix

[issue11395] print(s) fails on Windows with long strings

Changes by Santoso Wijaya : -- nosy: +santa4nt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue11395] print(s) fails on Windows with long strings

Amaury Forgeot d'Arc added the comment: Likewise, this fails with 3.2:: import os os.write(1, b"a" * 66000) -- nosy: +amaury.forgeotdarc ___ Python tracker ___ ___

[issue11395] print(s) fails on Windows with long strings

Changes by Amaury Forgeot d'Arc : -- nosy: +pitrou priority: normal -> high ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue11395] print(s) fails on Windows with long strings

New submission from Case Van Horsen : Python 3.2 fails when printing long strings. C:\Python32>python Python 3.2 (r32:88445, Feb 20 2011, 21:30:00) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print("a"*66000) Traceback (most re