[issue10224] Build 3.x documentation using python3.x
Sye van der Veen added the comment: I ran smack into this while setting up my Doc build for the first time. In trying to fix my build problems, it took me about an hour to find that this was a known issue. I've attached a patch to warn others of the issue, in the documentation and the build itself. Also, I've updated the version of Sphinx in building.rst, and made a note reminding others to do the same. I've only tested this patch on Windows 7. -- keywords: +patch nosy: +syeberman Added file: http://bugs.python.org/file22080/Issue10224Warnings.patch ___ Python tracker <http://bugs.python.org/issue10224> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10224] Build 3.x documentation using python3.x
Sye van der Veen added the comment: I had trouble wording that sentence. Under Unix, Makefile is used, in which you specify "PYTHON=" on the command line else the default "python" (from the PATH) is used. Under Windows, it's make.bat, where PYTHON needs to be set in the environment else "..\pcbuild\python" is used (which ignores the PATH). In either case, there's a high likelihood the PYTHON default will resolve to 3.3a0 for any Python developer. building.rst doesn't mention any of this, though; README.txt does. I wanted to keep the patch small, as I suspect this issue will be properly fixed soon, but I could create a new patch that better explains how to force that 2.x be used. -- ___ Python tracker <http://bugs.python.org/issue10224> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12179] Race condition using PyGILState_Ensure on a new thread
New submission from Sye van der Veen :
I'm wanting to call PyThreadState_SetAsyncExc from a function registered with
SetConsoleCtrlHandler. To do so, I need to call PyGILState_Ensure, which
asserts that Python is initialized, so I need to check for that. However, I
noticed a race condition with the code:
if( Py_IsInitialized( ) ) {
// XXX What if another thread calls Py_Finalize here?
gstate = PyGILState_Ensure( );
PyThreadState_SetAsyncExc( MainThreadId, PyExc_SystemExit );
PyGILState_Release( gstate );
}
What I need is to be able to hold the GIL around the entire block of code,
potentially before Py_Initialize is called for the first time. Now, 3.2
deprecated PyEval_AcquireLock, and PyEval_InitThreads can no longer be called
before Py_Initialize. Thankfully, I'm on 2.6.4, so I was able to write this
code:
PyEval_AcquireLock( );
if( Py_IsInitialized( ) ) {
gstate = PyGILState_Ensure( );
PyThreadState_SetAsyncExc( MainThreadId, PyExc_SystemExit );
PyGILState_Release( gstate );
}
PyEval_ReleaseLock( );
The problem in 2.6.4 is that PyGILState_Ensure deadlocks because the GIL is
already held, so that doesn't solve my problem. (Incidentally, the
PyGILState_Ensure docs say it works "regardless of the current state of the
GIL", which is incorrect.)
The 3.2 docs say to use PyEval_AcquireThread or PyEval_RestoreThread, which
both require an existing PyThreadState. To get that, I would need to call
PyThreadState_New, which needs a PyInterpreterState. To get _that_ I could use
PyInterpreterState_Head, since I know I only use one interpreter. Now I'm
getting into "advanced debugger" territory, but it's no use anyway; it's
possible that Py_Finalize could sneak in between the time I get this
interpreter and when I acquire the GIL, causing me to access a free'd
interpreter.
I believe the best fix for this would be to have a version of PyGILState_Ensure
that works even when Python is not initialized. It would not be able to create
a thread, and thus I would not expect to be able to call any Python API, but it
would always ensure the GIL is acquired. This _may_ have to be a new
"PyGILState_EnsureEx" function, because existing code expects PyGILState_Ensure
to always allow them to call the Python API. The resulting code would be:
gstate = PyGILState_EnsureEx( );
if( Py_IsInitialized( ) ) {
PyThreadState_SetAsyncExc( MainThreadId, PyExc_SystemExit );
}
PyGILState_Release( gstate );
This would require that Py_Initialize itself acquires the GIL.
The above problem was found on 2.6.4, but I've consulted the 3.2 docs and 3.3
code (via the online source) and it looks like the situation would be exactly
the same. In the meantime, I'm going to stick with the first piece of code and
hope nobody hits CTRL-BREAK during program clean-up.
--
components: Interpreter Core
messages: 136888
nosy: syeberman
priority: normal
severity: normal
status: open
title: Race condition using PyGILState_Ensure on a new thread
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
___
Python tracker
<http://bugs.python.org/issue12179>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10224] Build 3.x documentation using python3.x
Sye van der Veen added the comment: When I built the documentation on Win7, it failed with a SyntaxError, and it took some digging to find the reason why. I was hoping that issuing this warning would save others the trouble. I agree: on Windows, PYTHON is rarely set. However, look in make.bat: you'll see that when PYTHON is not already set, it's set to ..\pcbuild\python, which is Python 3. -- ___ Python tracker <http://bugs.python.org/issue10224> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18185] Error in test_set.TestVariousIteratorArgs.test_inline_methods
New submission from Sye van der Veen: I'm working with Python's test suite and noticed this error: http://hg.python.org/cpython/file/668aba845fb2/Lib/test/test_set.py#l1669 # etc... for g in (G, I, Ig, L, R): expected = meth(data) actual = meth(G(data)) # etc... I believe the assignment to actual should be using a lower-case 'g'. -- components: Tests messages: 190941 nosy: syeberman priority: normal severity: normal status: open title: Error in test_set.TestVariousIteratorArgs.test_inline_methods type: behavior ___ Python tracker <http://bugs.python.org/issue18185> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12488] multiprocessing.Connection does not communicate pipe closure between parent and child
Sye van der Veen added the comment: This issue _does_ exist on Windows, and is not limited to the case where the master process exits before its children. The following code, which is almost exactly that from the 2.7.3 documentation, deadlocks on Win7 (Py3.2 and 2.7) and WinXP (Py3.2 and 2.6): from multiprocessing import Process, Pipe import sys def f(conn): #conn.send([42, None, 'hello']) # uncomment second conn.close() if __name__ == "__main__": parent_conn, child_conn = Pipe() p = Process(target=f, args=(child_conn,)) p.start() #child_conn.close() # uncomment first sys.stdout.write( "about to receive\n" ) sys.stdout.write( "%s\n"%parent_conn.recv() ) sys.stdout.write( "received\n" ) p.join() If you "uncomment first", recv raises an EOFError; if you also "uncomment second", recv succeeds. If this behaviour is the same on other platforms, then it seems all that is required is to update the documentation. -- nosy: +syeberman versions: +Python 2.6 ___ Python tracker <http://bugs.python.org/issue12488> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16384] import.c doesn't handle EOFError from PyMarshal_Read*
New submission from Sye van der Veen:
The PyMarshal_Read* functions raise EOFError when the end of the file is
unexpectedly met. The current import.c functions propagate this error when
reading .pyc or .pyo files. One consequence of this is that Python will abort
on startup if, say, encodings.utf_8's .pyc file is truncated.
I have encountered a scenario where this truncation is common. If a second
Python process is launched while the first is writing the "core" .pyc files,
the second process may open the files before they are completely written and
will thus see truncated files and abort. This is a race condition that I was
able to reproduce consistently on several Windows Server 2008 RC2 Standard SP1
machines running 32-bit Python 3.2.3 from GNU make with "-j 16" (Intel Xeon
E5405 2GHz 2 processors 8GB 64-bit OS). (Of course, I had to clean the
__pycache__ directories between tests.)
This can be fixed in load_source_module by making read_compiled_module failures
non-fatal:
if (cpathname != NULL &&
(fpc = check_compiled_module(pathname, st.st_mtime, cpathname))) {
co = read_compiled_module(cpathname, fpc);
if (co == NULL) PyErr_Clear();
fclose(fpc);
}
if (co != NULL) {
// etc...
}
else {
co = parse_source_module(pathname, fp);
// etc...
write_compiled_module(co, cpathname, &st);
}
This is similar to how write_compiled_module ignores failures in writing the
.pyc files. It ensures that if the .pyc file is corrupt for _any_ reason, it
will get rewritten; this could be made specific to EOFError, but I don't
recommed that. Mostly, it ensures that corrupt .pyc files do not prevent
Python from loading if the .py files are valid.
--
components: None
messages: 174438
nosy: syeberman
priority: normal
severity: normal
status: open
title: import.c doesn't handle EOFError from PyMarshal_Read*
type: crash
versions: Python 3.2
___
Python tracker
<http://bugs.python.org/issue16384>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19023] ctypes docs: Unimplemented and undocumented features
New submission from Sye van der Veen: In the ctypes documentation, there's mention of a BigEndianUnion/LittleEndianUnion that isn't actually implemented, and the "Arrays and pointers" section just reads "Not yet written". The attached patch adds the (Big|Little)EndianUnion classes (with tests), finishes the Array/_Pointer class docs (and adds missing cases to the tests), and makes some stylistic improvements to docs. The patch was made against default, but it's quite likely it could be back-ported all the way to Python 3.1. -- assignee: docs@python components: Documentation, Tests, ctypes hgrepos: 209 messages: 197764 nosy: docs@python, syeberman priority: normal severity: normal status: open title: ctypes docs: Unimplemented and undocumented features type: enhancement ___ Python tracker <http://bugs.python.org/issue19023> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19023] ctypes docs: Unimplemented and undocumented features
Changes by Sye van der Veen : -- keywords: +patch Added file: http://bugs.python.org/file31769/75843d82f6cf.diff ___ Python tracker <http://bugs.python.org/issue19023> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue812750] OSA support for properties broken
Sye van der Veen added the comment: I've attached a patch to remove "a replacement is expected for Python 2.5". -- keywords: +patch nosy: +syeberman status: pending -> open Added file: http://bugs.python.org/file14715/Issue812750.diff ___ Python tracker <http://bugs.python.org/issue812750> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28937] str.split(): remove empty strings when sep is not None
Sye van der Veen added the comment:
In the sep!=None case, there are existing alternatives to prune=True that
aren't many more keystrokes:
>>> ''.split(' ', prune=True)
[]
>>> [x for x in ''.split(' ') if x]
[]
>>> list(filter(bool, ''.split(' '))) # or drop list() and use the iterator
>>> directly
[]
This becomes even fewer keystrokes for users that create a prune() or
split_prune() function.
For the sep==None case, I agree there are no alternatives to prune=False (aside
from rolling your own split function). However, instead of prune, what if sep
accepted a tuple of strings, similar to startswith. In this case, each string
would be considered one possible, yet distinct, delimiter:
>> ''.split(prune=False)
['']
>> ''.split((' ', '\t')) # typical whitespace
['']
>> ''.split(tuple(string.whitespace)) # ASCII whitespace
['']
Once again, this becomes even easier for users that create a split_no_prune()
function, or that assign tuple(string.whitespace) to a variable. It would also
nicely handle strings with non-homogeneous delimiters:
>>> '1?2,,3;'.split((',', ';', '?'))
['1', '2', '', '3', '']
I personally find the 0-argument str.split() one of the great joys of Python.
It's common to have to split out words from a sentence, and having that
functionality just 8 characters away at all times has been very useful.
--
nosy: +syeberman
___
Python tracker
<http://bugs.python.org/issue28937>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19023] ctypes docs: Unimplemented and undocumented features
Sye van der Veen added the comment: Signed and confirmed. :-) On Thu, Nov 12, 2015 at 11:28 PM Martin Panter wrote: > > Martin Panter added the comment: > > Sye van der Veen: can you sign a contributor agreement < > https://www.python.org/psf/contrib/contrib-form/>? I think it might be > needed for this patch. > > -- > > ___ > Python tracker > <http://bugs.python.org/issue19023> > ___ > -- ___ Python tracker <http://bugs.python.org/issue19023> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16384] import.c doesn't handle EOFError from PyMarshal_Read*
Sye van der Veen added the comment: I feel this patch (file44424) misses the mark. Any two Python processes that try to import a module, without a pyc, at the same time could suffer race conditions. The first process will start to write the pyc, get interrupted, and the second will fail with an EOFError. When importing encodings at startup, this is a nasty abort. But it's just as nasty if a running script gets unpredictable EOFErrors. Corrupt .pyc files should not prevent importing if the valid .py files are available. On Tue, Sep 6, 2016, 11:29 PM Eric Snow wrote: > > Eric Snow added the comment: > > After looking more closely, it looks like we should be ignoring such bogus > modules. Here's a patch to do so. > > -- > keywords: +patch > stage: test needed -> patch review > Added file: http://bugs.python.org/file44424/issue16384.diff > > ___ > Python tracker > <http://bugs.python.org/issue16384> > ___ > -- ___ Python tracker <http://bugs.python.org/issue16384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28007] Bad .pyc files prevent import of otherwise valid .py files.
Sye van der Veen added the comment: Consider a process that is terminated while writing a large .pyc file. Currently a user can only fix this by deleting the .pyc file, which requires knowing where to look. A developer can be expected to know this, but the end user of their application certainly would not. .pyc files are a cache. When your cache is corrupted, you recreate it. -- ___ Python tracker <http://bugs.python.org/issue28007> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28005] Broken encoding modules are silently skipped.
Changes by Sye van der Veen : -- nosy: +syeberman ___ Python tracker <http://bugs.python.org/issue28005> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16384] import.c doesn't handle EOFError from PyMarshal_Read*
Sye van der Veen added the comment: I would also agree that failing to load the main codec should be an abort. This bug was specifically the race condition in writing the .pyc file. Thanks for all your help! -- ___ Python tracker <http://bugs.python.org/issue16384> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
