[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Guido van Rossum wrote: > You might want to compare what I checked in to your patch; I did a few > style cleanups. I also moved the lseek() call into import.c, where it > seems more appropriate. Ah I see that you prefer to keep assignment and check against NU

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: OK, checked in. You might want to compare what I checked in to your patch; I did a few style cleanups. I also moved the lseek() call into import.c, where it seems more appropriate. Committed revision 58587. -- resolution: -> accepted status: open -

[issue1267] Py3K cannot run as ``python -S``

Changes by Guido van Rossum: -- resolution: accepted -> status: closed -> open __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing l

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: > > Do you have access to Windows? I believe it doesn't have dup(). :-( > > I've an old laptop with Win2k at my disposal but it has no VS yet. Can > you point me to a set of instruction how to install VS 2003? I've just > VS .NET 2005 available. Sorry, I'm as

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Guido van Rossum wrote: > Guido van Rossum added the comment: > > Do you have access to Windows? I believe it doesn't have dup(). :-( I've an old laptop with Win2k at my disposal but it has no VS yet. Can you point me to a set of instruction how to install VS

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: Do you have access to Windows? I believe it doesn't have dup(). :-( __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: > I think find_module() should return a file descriptor (fd), not a > FILE*, but most of the rest of the code should call fdopen() on that > fd. Only call_find_module() should use the fd to turn it into a Python > file object. Then the amount of change should b

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: > Is PyFile_FromFd and PyFile_FromFdEx fine with you or do you prefer a > different name like PyFile_FromFD and PyFile_FromFDEx or > PyFile_FromFileDescriptor? Let's have a single function PyFile_FromFd(fd, name, mode, buffering, encoding, newline). > I've an

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: > The dangers of switching between fileno(fp) and fp are actually well > documented in the C and/or POSIX standards. The problem is caused in > PyFile_FromFileEx() -- it creates a Python file object from the file > descriptor. The fix actually only works becaus

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Guido van Rossum wrote: > I think find_module() should return a file descriptor (fd), not a > FILE*, but most of the rest of the code should call fdopen() on that > fd. Only call_find_module() should use the fd to turn it into a Python > file object. Then the a

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: > Guido, Brett, how much of the code should I change? import.c is mainly > using file pointers. Should I replace all operations on FILE with > operations on a file descriptor? I think find_module() should return a file descriptor (fd), not a FILE*, but most of

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: > Yes, I think you got it. Guido, Brett, how much of the code should I change? import.c is mainly using file pointers. Should I replace all operations on FILE with operations on a file descriptor? Christian __ Tracker <[EMAIL

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: On 10/20/07, Christian Heimes <[EMAIL PROTECTED]> wrote: > > Christian Heimes added the comment: > > Guido van Rossum wrote: > > You're right that a lot of this could be avoided if we used file > > descriptors consistently. It seems find_module() itself doesn't rea

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Guido van Rossum wrote: > You're right that a lot of this could be avoided if we used file > descriptors consistently. It seems find_module() itself doesn't read > the file; it just needs to know that it's possible to open the file. > > Rewriting everywhere th

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: Sorry, I was wrong about the encoding leak; you fixed that. The FILE/fd issue is real though. __ Tracker <[EMAIL PROTECTED]> __ _

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: Thanks for persevering!!! The dangers of switching between fileno(fp) and fp are actually well documented in the C and/or POSIX standards. The problem is caused in PyFile_FromFileEx() -- it creates a Python file object from the file descriptor. The fix actuall

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Brett Cannon wrote: > Thanks. I just wish this whole ordeal had not been necessary (filed a > bug report with Apple in hopes that this can be prevented for someone > else). I can see why some people prefer to hack on PyPy, IronPython, > or Jython instead of d

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: On 10/19/07, Christian Heimes <[EMAIL PROTECTED]> wrote: > > Christian Heimes added the comment: > > Brett Cannon wrote: > > This suggests that perhaps we should standardize on file pointers or > > file descriptors in Python to prevent something like this from happ

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Brett Cannon wrote: > This suggests that perhaps we should standardize on file pointers or > file descriptors in Python to prevent something like this from happening > again. rewind() it used couple of times in the Python code. Have you checked if the other ca

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: 58557 as the fix. Yes, it was stupid on OS X's part and I was lucky to just think of the possible solution. Basically OS X does not like it when you do stuff with a file pointer but then poke around with the file descriptor. That means that when PyTokenizer_Find

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: > OK, I think I might have a solution, and it is really stupid. I will > run the unit test suite to see if it really fixes everything. Here's keeping my fingers crossed. I do note that the new code still leaks the encoding string which is malloc'ed in PyToke

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: OK, I think I might have a solution, and it is really stupid. I will run the unit test suite to see if it really fixes everything. __ Tracker <[EMAIL PROTECTED]> __

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: Nope, didn't do it. I also checked using gdb a few minutes ago and ftell() was reporting a position of 0 all they way back to PyObject_MethodCall(). __ Tracker <[EMAIL PROTECTED]>

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: I checked in your ``< 2`` change and plugged a memory leak since you were not freeing the struct tok_state. Checked in r58555. __ Tracker <[EMAIL PROTECTED]> __ ___

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Brett Cannon wrote: > Brett Cannon added the comment: > > OK, for some reason, when PyTokenizer_FindEncoding() is called and the > resulting file is big enough, it starts off with a seek position > (according to TextIOWrapper.tell()) of 4096. That happens to

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: I've made a short unit tests which tests a large file with and w/o -*- coding: -*-. It passes on Linux. Added file: http://bugs.python.org/file8575/py3k_test_issue1267.patch __ Tracker <[EMAIL PROTECTED]>

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: OK, for some reason, when PyTokenizer_FindEncoding() is called and the resulting file is big enough, it starts off with a seek position (according to TextIOWrapper.tell()) of 4096. That happens to be exactly half the size of the buffer used by io. But I am not su

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: It's unrelated to the problem but Parser/tokenizer.c:1619 has a minor bug. > while(((tok->lineno <= 2) && (tok->done == E_OK))) { > PyTokenizer_Get(tok, &p_start, &p_end); > } (tok->lineno < 2) is sufficient. See line 516 Chri

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: PyTokenizer_FindEncoding() is the culprit. If I comment it out in Python/import.c:call_find_module(), and thus just use the system encoding, runpy works again. __ Tracker <[EMAIL PROTECTED]> _

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Brett Cannon wrote: > Brett Cannon added the comment: > > It looks like the file object returned by imp.find_module() has its read > position WAY too far forward (at least on OS X). That's strange. It should never read more than 2 lines of a file. I don't und

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: > runpy is failing because pkgutil is failing because it is giving > compile() part of a source file instead of the entire thing:: > > Traceback (most recent call last): > File "/Users/drifty/Dev/python/3.x/pristine/Lib/runpy.py", line 97, > in _run_module_a

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: It looks like the file object returned by imp.find_module() has its read position WAY too far forward (at least on OS X). Re-opening. -- status: closed -> open __ Tracker <[EMAIL PROTECTED]>

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: runpy is failing because pkgutil is failing because it is giving compile() part of a source file instead of the entire thing:: Traceback (most recent call last): File "/Users/drifty/Dev/python/3.x/pristine/Lib/runpy.py", line 97, in _run_module_as_main loade

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: I don't have a Mac at my disposal any more. :( Can you attach the output of the failing tests to the bug report? Maybe I can be of assistance. __ Tracker <[EMAIL PROTECTED]> __

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: Yeah, I just noticed this as well. is it definitely this change? On 10/19/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > Guido van Rossum added the comment: > > FWIW, on Mac I get six failures (all these pass on Linux): > > test_cmd_line test_inspect test_mo

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: FWIW, on Mac I get six failures (all these pass on Linux): test_cmd_line test_inspect test_modulefinder test_pyclbr test_quopri test_runpy __ Tracker <[EMAIL PROTECTED]> __

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: Committed revision 58553 (with minor tweaks only). Thanks! -- resolution: -> accepted status: open -> closed __ Tracker <[EMAIL PROTECTED]> __ ___

[issue1267] Py3K cannot run as ``python -S``

Brett Cannon added the comment: Thanks, Guido. __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: I will take this. -- assignee: brett.cannon -> gvanrossum __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Update since last patch * removed unnecessary const from const char* PyTokenizer_FindEncoding(FILE *fp) * Fixed bug in find_module whith binary files Please review the patch. __ Tracker <[EMAIL PROTECTED]>

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: Does this mean I should hold off reviewing the patch? On 10/16/07, Christian Heimes <[EMAIL PROTECTED]> wrote: > > Christian Heimes added the comment: > > Christian Heimes wrote: > > * removed unused import of open in initstdio() > > * fixed infinite loop in

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Christian Heimes wrote: > * removed unused import of open in initstdio() > * fixed infinite loop in PyTokenizer_FindEncoding() by checking > tok-done == E_OK I found another bug in Python/import.c:call_find_method. The function mustn't set an encoding of ftp

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Changes since last patch: * removed unused import of open in initstdio() * fixed infinite loop in PyTokenizer_FindEncoding() by checking tok-done == E_OK __ Tracker <[EMAIL PROTECTED]> _

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: The patch is a combined patch for the imp.find_module() problem and initstdio. Both patches depend on the new PyFile_FromFileEx() function. I hope you don't mind. __ Tracker <[EMAIL PROTECTED]>

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: Guido van Rossum wrote: > Don't call open() with keyword arg for newline="\r"; open() takes > positional args too. This is done specifically to simplify life for C > code calling it. :-) Perhaps one of the PyFunction_Call(..) variants > makes it easier to call

[issue1267] Py3K cannot run as ``python -S``

Guido van Rossum added the comment: > I've carefully checked and tested the initstdio() method. I'm sure that > I've catched every edged case. The unit tests pass w/o complains. > > I've also added a PyErr_Display() call to Py_FatalError(). It's still > hard to understand an error in io.py but at

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: I've carefully checked and tested the initstdio() method. I'm sure that I've catched every edged case. The unit tests pass w/o complains. I've also added a PyErr_Display() call to Py_FatalError(). It's still hard to understand an error in io.py but at least th

[issue1267] Py3K cannot run as ``python -S``

Changes by Brett Cannon: -- assignee: -> brett.cannon keywords: +patch __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing list Uns

[issue1267] Py3K cannot run as ``python -S``

Christian Heimes added the comment: I've some code around which sets sys.stdin, out and err in C code. The code is far from perfect and I haven't checked it for reference leaks yet. I like to get your comment on the style and error catching. -- nosy: +tiran _

[issue1267] Py3K cannot run as ``python -S``

Changes by Guido van Rossum: -- nosy: +gvanrossum __ Tracker <[EMAIL PROTECTED]> __ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue1267] Py3K cannot run as ``python -S``

New submission from Brett Cannon: If Py3K is executed without importing site, it fails horribly. This is because site.py sets __builtins__.open, sys.stdout, sys.stderr, and sys.stdin. The setting of sys.stderr is especially bad as exception printing requires sys.stderr, otherwise it reports tha