[issue38787] PEP 573: Module State Access from C Extension Methods
New submission from Marcel Plch : Currently, there is not way to access per-module state from the methods of types defined in extension modules. This PEP provides a fix to this issue. PEP 573 - https://www.python.org/dev/peps/pep-0573/ Reference implementation - https://github.com/Dormouse759/cpython/tree/pep-c-rebase_newer -- components: Extension Modules messages: 356533 nosy: Dormouse759, ncoghlan, petr.viktorin, scoder, terry.reedy priority: normal severity: normal status: open title: PEP 573: Module State Access from C Extension Methods type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue38787> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38787] PEP 573: Module State Access from C Extension Methods
Change by Marcel Plch : -- keywords: +patch pull_requests: +16655 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17145 ___ Python tracker <https://bugs.python.org/issue38787> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39460] test_zipfile: test_add_file_after_2107() fails on s390x Fedora Rawhide 3.x
Marcel Plch added the comment: Is currently anybody actively working on this? Please, report what you have found out, if so. I'd like to start digging into this tomorrow and possibly avoid any duplicit work. -- ___ Python tracker <https://bugs.python.org/issue39460> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39761] Python 3.9.0a4 fails to build when configured with --with-dtrace
New submission from Marcel Plch : Steps to reproduce: $ wget https://www.python.org/ftp/python/3.9.0/Python-3.9.0a4.tar.xz $ tar xvf Python-3.9.0a4.tar.xz $ cd Python-3.9.0a4 $ ./configure --with-dtrace $ make -j12 /usr/bin/ld: libpython3.9.a(ceval.o): in function `_PyEval_EvalFrameDefault': /home/mplch/Work/fedpkg/Python-3.9.0a4/Python/ceval.c:1117: undefined reference to `python_function__entry_semaphore' /usr/bin/ld: /home/mplch/Work/fedpkg/Python-3.9.0a4/Python/ceval.c:1254: undefined reference to `python_line_semaphore' /usr/bin/ld: /home/mplch/Work/fedpkg/Python-3.9.0a4/Python/ceval.c:3697: undefined reference to `python_function__return_semaphore' /usr/bin/ld: /home/mplch/Work/fedpkg/Python-3.9.0a4/Python/ceval.c:1445: undefined reference to `python_line_semaphore' ... /usr/bin/ld: libpython3.9.a(gcmodule.o):(.note.stapsdt+0x70): undefined reference to `python_gc__done_semaphore' collect2: error: ld returned 1 exit status make: *** [Makefile:709: Programs/_testembed] Error 1 Additional info: $ gcc --version gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1) Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -- components: Build messages: 362700 nosy: Dormouse759 priority: normal severity: normal status: open title: Python 3.9.0a4 fails to build when configured with --with-dtrace type: compile error versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue39761> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST
New submission from Marcel Plch : Problem: If you want to override CFLAGS by setting EXTRA_CFLAGS, they may have no effect if there are contrary flags in the CFLAGS_NODIST variable. How to reproduce: make CFLAGS_NODIST="-O2" EXTRA_CFLAGS="-Og" If you look at GCC arguments, there is -O2 present *after* the -Og flag. This means -Og gets ignored. -- components: Build messages: 348168 nosy: Dormouse759 priority: normal severity: normal status: open title: EXTRA_CFLAGS get overrided by CFLAGS_NODIST type: behavior versions: Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue37631> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37631] EXTRA_CFLAGS get overrided by CFLAGS_NODIST
Marcel Plch added the comment: I believe you mean this downstream issue: https://bugzilla.redhat.com/show_bug.cgi?id=1712977 That issue is but only a consequence of a bad flag handling. The bad flag handling affects not only test_gdb on that specific architecture, but the entire optimization level on all architectures, hence causing inconveniences in debugging in general on all architectures. It's a pure chance that one test caught this specific case. It might also cause inconveniences with other use-cases for EXTRA_CFLAGS, as they might get overridden by CFLAGS_NODIST. You can get the erroneous output by running the reproducer. That is: $./configure ... $ make CFLAGS_NODIST="-O2" EXTRA_CFLAGS="-Og" gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Og -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -O2 -I./Include/internal -I. -I./Include-DPy_BUILD_CORE -o Programs/python.o ./Programs/python.c -Og from EXTRA_CFLAGS gets overridden by -O2 from CFLAGS_NODIST. -- ___ Python tracker <https://bugs.python.org/issue37631> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30403] PEP 547: Running extension modules using -m switch
Marcel Plch added the comment: Sorry for not responding for so long, I didn't work on Python through the summer because of some other matters. Re-execution of the module is not possible, because there is a check on the C level, If you call exec_in_module() on an already initialized module, it'll raise ImportError. Also, because of this check, there is the restriction for py_mod_create. "Modules" defining this slot might not be module objects at all, so they might not have the module state pointer (which is used to flag if the module was initialized). -- ___ Python tracker <http://bugs.python.org/issue30403> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30403] PEP 547: Running extension modules using -m switch
Marcel Plch added the comment: I have made a patch both for cython and cpython implementing a way to use Py_mod_create in cython. Basically module def that specifies a new "Py_mod_cython" slot are excluded from the rule of no module creation, so these modules can be executed directly even though they specify Py_mod_create. Is this approach safe or does it make easy for things to go wrong? cpython - https://github.com/Traceur759/cpython/commit/51a7508d176b23dcf3547b970cf7e6a50898aae2 cython - https://github.com/Traceur759/cython/commit/2ca706e10f469cd38947eecd8b92c142532b20bc -- ___ Python tracker <https://bugs.python.org/issue30403> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30177] pathlib.resolve(strict=False) only includes first child
Changes by Marcel Plch : -- pull_requests: +1744 ___ Python tracker <http://bugs.python.org/issue30177> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30403] Running extension modules using -m switch
New submission from Marcel Plch: Currently the -m switch does not work with extension modules: $ python3 -m math /usr/bin/python3: No code object available for math In order to enable extension modules to behave like Python source modules, the -m switch should be supported. Please, see this proof of concept: https://github.com/Traceur759/cpython/tree/main_c_modules -- components: Extension Modules messages: 293954 nosy: Dormouse759 priority: normal severity: normal status: open title: Running extension modules using -m switch type: enhancement versions: Python 3.7 ___ Python tracker <http://bugs.python.org/issue30403> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30403] Running extension modules using -m switch
Changes by Marcel Plch : -- pull_requests: +1845 ___ Python tracker <http://bugs.python.org/issue30403> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19717] resolve() fails when the path doesn't exist
Changes by Marcel Plch : -- pull_requests: +1930 ___ Python tracker <http://bugs.python.org/issue19717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29673] Some gdb macros are broken in 3.6
Change by Marcel Plch : -- keywords: +patch pull_requests: +5888 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue29673> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29673] Some gdb macros are broken in 3.6
Marcel Plch added the comment: I have created a PR here - https://github.com/python/cpython/pull/6126 The problem was, indeed, change in the code structure. The macro checked for presence inside of PyEval_EvalFrame() using address of a neighbouring function. Also, arguments to PyEval_EvalFrame() were changed, so the macro responsible for printing of the frame needed a little tweak. -- nosy: +Dormouse759 ___ Python tracker <https://bugs.python.org/issue29673> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32962] test_gdb fails in debug build with `-mcet -fcf-protection -O0`
Change by Marcel Plch : -- keywords: +patch pull_requests: +6441 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue32962> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31862] Port the standard library to PEP 489 multiphase initialization
New submission from Marcel Plch : PEP 489 introduced multiphase initialization of extension and built-in modules. Now, almost no module in the standard library supports this feature. This should be improved to prepare Python for better testing of subinterpreters. Many benefits of PEP 489 don't apply to stdlib modules. However, the PEP effectively says that by using multi-phase init, the module author "promises" that the module is "subinterpreter-friendly" [0]. So, when porting, each module should be checked that it e.g. doesn't use mutable process-global state. I'd like to port stdlib to multi-phase init, starting with the easier modules, to: - get familiar with contributing to CPython, - check and track which modules are already "subinterpreter-friendly", and - figure out how and where PEP 489 is lacking (beyond what is discussed in the PEP itself). [0]: https://www.python.org/dev/peps/pep-0489/#subinterpreters-and-interpreter-reloading -- components: Extension Modules messages: 304914 nosy: Dormouse759, encukou, ncoghlan priority: normal severity: normal status: open title: Port the standard library to PEP 489 multiphase initialization type: enhancement versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31862] Port the standard library to PEP 489 multiphase initialization
Change by Marcel Plch : -- keywords: +patch pull_requests: +4078 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31862> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31901] atexit callbacks only called for current subinterpreter
New submission from Marcel Plch : `Py_FinalizeEx()` only executes callbacks from the subinterpreter from which Py_FinalizeEx is called. What is the expected behavior here? Should the callbacks be specific to individual subinterpreters? -- components: Extension Modules messages: 305233 nosy: Dormouse759, encukou, ncoghlan priority: normal severity: normal status: open title: atexit callbacks only called for current subinterpreter type: behavior versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue31901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31901] atexit callbacks should be run at subinterpreter shutdown
Change by Marcel Plch : -- keywords: +patch pull_requests: +4527 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue31901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31901] atexit callbacks should be run at subinterpreter shutdown
Marcel Plch added the comment: I created a PR with fix on this issue - https://github.com/python/cpython/pull/4611 This makes Py_EndInterpreter() call atexit callbacks for the subinterpreter it is destroying. It doesn't make Py_Finalize() end all subinterpreters, as the current implementation of subinterpreters makes it hard to do so. This is the same as the current behaviour: you need to end all subinterpreters before calling Py_Finalize(). -- ___ Python tracker <https://bugs.python.org/issue31901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6531] atexit_callfuncs() crashing within Py_Finalize() when using multiple interpreters.
Marcel Plch added the comment: > At a guess, the problem is because in atexit_callfuncs(): > >module = PyState_FindModule(&atexitmodule); >if (module == NULL) >return; In #31901, I solved this problem by getiing rid of PyState_FindModule in the atexit code, using module state and PEP-489 multiphase initialization. This needed some changes in the pystate and pylifecycle code, so the specific interpreter has a reference to its own atexit module. This way, Py_EndInterpreter() is able to call the callbacks for the given interpreter. See: https://github.com/python/cpython/pull/4611 -- ___ Python tracker <https://bugs.python.org/issue6531> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32374] Document that m_traverse for multi-phase initialized modules can be called with m_state=NULL
Change by Marcel Plch : -- keywords: +patch pull_requests: +4821 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue32374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32374] Document that m_traverse for multi-phase initialized modules can be called with m_state=NULL
Change by Marcel Plch : -- pull_requests: +4999 ___ Python tracker <https://bugs.python.org/issue32374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32374] Document that m_traverse for multi-phase initialized modules can be called with m_state=NULL
Marcel Plch added the comment: I have created a PR at https://github.com/python/cpython/pull/5140 It contains documentation, plus, in --with-pydebug mode, it calls m_traverse to catch easy cases of not handling the m_state==NULL case early. -- ___ Python tracker <https://bugs.python.org/issue32374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32374] Document that m_traverse for multi-phase initialized modules can be called with m_state=NULL
Change by Marcel Plch : -- pull_requests: +6779 ___ Python tracker <https://bugs.python.org/issue32374> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO
Marcel Plch added the comment: LTO may break the debug symbols and make GDB unusable. There is an option, that fixes the issue: to use a -g switch in link flags. Note that this slows loading of the debug symbols significantly. I suggest these options as possible approaches: 1) make the configure script include -g in LDFLAGS when --enable-optimizations and --with-lto are used 2) same as 1), but only when --with-pydebug is also used. 3) document this problem and make the user aware that this possible fix (-g in link flags) exists -- nosy: +Dormouse759 ___ Python tracker <https://bugs.python.org/issue30345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO
Marcel Plch added the comment: Yes, but that is not a fix really in this case. While it makes the test pass because it 'correctly' prints out unknown objects, it makes no real difference when actually debugging. The -g switch at link time makes the debug symbols readable and user is able to debug just as usual. -- ___ Python tracker <https://bugs.python.org/issue30345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO
Marcel Plch added the comment: Those -g switches you see there are during compile-time. For this to work, you need to enable it also during link/time: ./configure --enable-optimizations --with-lto LDFLAGS="-g" Except for py-bt, you should also try bt. With this link flag enabled, I can observe significant slowdown on my machine during the backtrace when using bt command (At least when I let the PGO do all the profiling, when compiled with the sed edit you posted here, I observe none). -- ___ Python tracker <https://bugs.python.org/issue30345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO
Change by Marcel Plch : -- nosy: +petr.viktorin ___ Python tracker <https://bugs.python.org/issue30345> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32962] test_gdb fails in debug build with `-mcet -fcf-protection -O0`
Marcel Plch added the comment:
The problem is with this function:
static PyObject *
builtin_id(PyModuleDef *self, PyObject *v)
/*[clinic end generated code: output=0aa640785f697f65 input=5a534136419631f4]*/
{
return PyLong_FromVoidPtr(v);
}
It's a one-liner, so the compiler really likes to inline it.
Without the inline optimization, the additional "next" command makes a jump
into the function.
But when the function is inlined and you set a breakpoint to it, the line is
just seen as a function from the debugger, that means you already are inside
and the "next" makes the debugger exit this line, and so the function.
More graphical explanation:
non-inline case:
br
{
next
return PyLong_FromVoidPtr(v);
inline case:
br
return PyLong_FromVoidPtr(v);
next
"Some code without access to the func arguments' debug symbols"
I propose two possible solutions:
1) Skip whole test_gdb when optimizations are used (who debugs with them
anyway?)
2) Conditionalize the "next". (this could be hard as we would need to know when
the function is inlined)
Also, I have found out that when configured with --with-pydebug and
--enable-optimizations, tests stop to fail. (the failing bots are configuring
with --enable-optimizations only)
--
___
Python tracker
<https://bugs.python.org/issue32962>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32962] test_gdb fails in debug build with `-mcet -fcf-protection -O0`
Marcel Plch added the comment: > It's already done, no? But the title issue is "-mcet -fcf-protection > -O0" and -O0 disables optimizations. Some of the simple tests are still run even with optimizations. Disabled optimizations is what we want, because then the function doesn't get inlined --> the 'next' jumps in, not out. -- ___ Python tracker <https://bugs.python.org/issue32962> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34097] ZIP does not support timestamps before 1980
Marcel Plch added the comment: I'm going to have a closer look at this and try to add the option. -- nosy: +Dormouse759 ___ Python tracker <https://bugs.python.org/issue34097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34097] ZIP does not support timestamps before 1980
Change by Marcel Plch : -- keywords: +patch pull_requests: +7805 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34097] ZIP does not support timestamps before 1980
Marcel Plch added the comment: I have created a PR for this: https://github.com/python/cpython/pull/8270 -- ___ Python tracker <https://bugs.python.org/issue34097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34097] ZIP does not support timestamps before 1980
Marcel Plch added the comment: It seems reasonable, I'll have a look at it. -- ___ Python tracker <https://bugs.python.org/issue34097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34325] test_zipfile gets OverflowError in multiple buildbots
Change by Marcel Plch : -- keywords: +patch pull_requests: +8151 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue34325> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34097] ZIP does not support timestamps before 1980
Change by Marcel Plch : -- pull_requests: +8209 stage: resolved -> patch review ___ Python tracker <https://bugs.python.org/issue34097> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34566] pipe read sometimmes returns EOF but returncode is still None
New submission from Marcel Plch : TL;DR: For reproducer, please see attached file and the end of this description for a runner script. It seems that when pipe is being read it has a chance of returning EOF and not setting the return code. This bug affects all (or at least a broad set of) architectures and is present in all versions. (2.7 and 3.8 were tested). This bug is not reproducible when configured using --with-pydebug flag. As a result, any code relying on proper setting of the returncode attribute might (and probably is going to) fail. --- $ for i in $(seq 1 1000); do ./python script.py; done | grep poll | sort | uniq -c Actual output: 630 output: '', return code: 0, pollstatus=0 370 output: '', return code: None, pollstatus=None Expected output: 1000 output: '', return code: 0, pollstatus=0 -- components: Library (Lib) files: script.py messages: 324508 nosy: Dormouse759 priority: normal severity: normal status: open title: pipe read sometimmes returns EOF but returncode is still None versions: Python 3.8 Added file: https://bugs.python.org/file47782/script.py ___ Python tracker <https://bugs.python.org/issue34566> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34566] pipe read sometimes returns EOF but returncode is still None
Marcel Plch added the comment: original downstream issue - https://bugzilla.redhat.com/show_bug.cgi?id=1623070 -- title: pipe read sometimmes returns EOF but returncode is still None -> pipe read sometimes returns EOF but returncode is still None ___ Python tracker <https://bugs.python.org/issue34566> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34566] pipe read sometimes returns EOF but returncode is still None
Change by Marcel Plch : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue34566> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
