[issue39819] NULL pointer crash in Modules/_cursesmodule.c in PyInit__curses() on MIPS uclibc-ng and ncurses-6.2
New submission from Joshua Kinard :
Inside a MIPS O32 chroot, based on uclibc-ng-1.0.32, if python-27 or python-3.7
are built against ncurses-6.2, then after compilation, there is a crash in the
'_curses' module. Specific to Python-3.7, the crash is in
Modules/_cursesmodule.c:3482, PyInit__curses():
3477:{
3478:int key;
3479:char *key_n;
3480:char *key_n2;
3481:for (key=KEY_MIN;key < KEY_MAX; key++) {
3482:key_n = (char *)keyname(key);
3483:if (key_n == NULL || strcmp(key_n,"UNKNOWN KEY")==0)
3484:continue;
3485:if (strncmp(key_n,"KEY_F(",6)==0) {
3486:char *p1, *p2;
It looks like keyname() is casting to a NULL pointer and crashing when 'key' is
257. The issue is reproducible by running python and trying to import the
curses modules:
# python
Python 3.7.6 (default, Feb 29 2020, 22:51:27)
[GCC 9.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import curses
Segmentation fault
Or:
# python -m curses
Segmentation fault
dmesg shows this on the main host:
[99297.243445] do_page_fault(): sending SIGSEGV to python for invalid read
access from
[99297.243459] epc = in python3.7m[40+1]
[99297.243483] ra = 76a68c6c in
_curses.cpython-37m-mips-linux-gnu.so[76a5+2]
I've been able to work out that the fault has something to do with ncurses
itself. There is no issue if built against ncurses-6.1, and even the later
datestamped patches appear to be okay. It seems like any ncurses AFTER
20190609 will exhibit the problem. ncurses-6.2 was recently released, and it,
too, causes this issue.
However, I am unable to get gdb to trace through any of the ncurses libraries.
The faulting code is in Python itself, so I assume it's something to do with a
macro definition or an include provided by ncurses-6.2 that introduces the
breakage.
This issue also only happens in a uclibc-ng-based root. I have had zero issues
building python-3.7 in multiple glibc-based roots and even a musl-1.1.24-based
root works fine. So I am not completely sure if the fault is truly with Python
itself, or the combination of uclibc-ng, ncurses-6.2, and Python.
As far as I know, the issue may also be specific to MIPS hardware, but I do not
have a similar chroot on any other architecture to verify this with. I'll
attach to this bug a gdb backtrace of Python built with -O0 and -gddb3. I have
a core file available if that will help, but will probably need to e-mail that
as I'll have to include the malfunctioning python binary and the separate debug
symbol files generated from my build.
--
components: Extension Modules
files: py37-gdb-bt-sigsegv-cursesmodule-uclibc-20200301.txt
messages: 363107
nosy: kumba
priority: normal
severity: normal
status: open
title: NULL pointer crash in Modules/_cursesmodule.c in PyInit__curses() on
MIPS uclibc-ng and ncurses-6.2
type: crash
versions: Python 2.7, Python 3.7
Added file:
https://bugs.python.org/file48942/py37-gdb-bt-sigsegv-cursesmodule-uclibc-20200301.txt
___
Python tracker
<https://bugs.python.org/issue39819>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32328] ttk.Treeview: _tkinter.TclError: list element in quotes followed by "; " instead of space
Joshua Kinard added the comment: I have to disagree rather strongly with closing this as "not a bug". If the issue really boils down to incorrectly passing a single string when there is only one item in 'values', and a single-item tuple should be used instead, then shouldn't the better solution be for Python to handle this automatically and re-cast the errant string into a tuple? My example indicates a case of where I got the Tcl interpreter itself to choke on invalid user input. Even though the tkinter binding module catches the exception from the Tcl interpreter, I think it's Python's job as the wrapping language to catch any user stupidity done in Python and try to work around it if it's simple to do so. A user coding against Tkinter/Ttk should not be required to have or need knowledge of Tcl/Tk itself to code a GUI. That knowledge certainly helps, but not having it is one of Python's selling points for using Tkinter in the first place. I have a coding project where I initially tripped up the semi-colon issue while parsing data from a text file into a Treeview widget (in listbox mode), that has a single-column. The data being parsed included quoted regular expression syntax. The function doing the parsing passed the parsed data to another function that populated the Treeview using **kwargs to send "values" and "tags" to Treeview's "insert" method. It was a backslash character in the quoted regular expression string, at a point before the semi-colon, that eventually goofed the Tcl interpreter up, but //only// when passed in via **kwargs syntax. The original code was too complex to replicate for the bug, so I reduced things down to a smaller testcase. During this, I inadvertently tripped up the other bug with the unmatched open quote error, so I reported both. My current workaround involves scanning the input for a backslash character or a whitespace character (which also trips things up), and if found, wrap the string in curly braces, which disables Tcl's substitution mechanism for that string, then calling Treeview's "insert" method. It's more complicated this way, but safe for dealing with regular expression strings as possible input. -- ___ Python tracker <https://bugs.python.org/issue32328> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32328] ttk.Treeview: _tkinter.TclError: list element in quotes followed by "; " instead of space
New submission from Joshua Kinard :
I think I've found another bug with Tkinter's string handling in Python 2.7.14,
before it passes off to the TCL interpreter when inserting items into a
ttk.Treeview widget. Specifically, it doesn't handle strings containing
semi-colons very well if they are passed in via a kwarg dict.
Below is a simple test case. The "kw" variable has a 'values' key with a
string value using single quotes to store the value '" ";':
---
from Tkinter import *
from ttk import *
root = Tk()
tv = Treeview(root, columns=("foobar",))
tv.heading("foobar", text="Foobar!")
tv.column("#0", width=0)
tv.column("foobar", stretch=True, minwidth=100, width=400)
tv.grid()
kw = {'values': '" ";', 'tags': ''}
tv.insert("", END, iid=None, **kw)
root.mainloop()
---
This sample triggers this traceback:
Traceback (most recent call last):
File "test.py", line 14, in
tv.insert("", END, iid=None, **kw)
File "C:\Python27\lib\lib-tk\ttk.py", line 1339, in insert
res = self.tk.call(self._w, "insert", parent, index, *opts)
_tkinter.TclError: list element in quotes followed by ";" instead of space
Furthermore, if the 'values' key is changed to the string 'X:"Y ";', you
trigger a different traceback:
Traceback (most recent call last):
File "test.py", line 13, in
tv.insert("", END, iid=None, **kw)
File "C:\Python27\lib\lib-tk\ttk.py", line 1339, in insert
res = self.tk.call(self._w, "insert", parent, index, *opts)
_tkinter.TclError: unmatched open quote in list
So definitely a case of something in the parser not handling things properly
before sending to the TCL interpreter.
I suspect this is similar to Issue #15861. One can work around this bug by
checking for either a whitespace or backslash character in the string, and
wrapping the string in curly braces if so, to disable TCL substitution
mechanism, and it'll insert the string and display it correctly in the widget.
I have only verified this in Python 2.7.14 at the moment, which is what I
primarily work in right now.
--
components: Tkinter
messages: 308340
nosy: kumba
priority: normal
severity: normal
status: open
title: ttk.Treeview: _tkinter.TclError: list element in quotes followed by ";"
instead of space
type: behavior
versions: Python 2.7
___
Python tracker
<https://bugs.python.org/issue32328>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18597] On Windows sys.stdin.readline() doesn't handle Ctrl-C properly
Joshua Kinard added the comment: I was able to modify eryksun's patch for Python-2.7.15, at least the bulk of it. It looks like the '_PyOS_SigintEvent' function is new to the 3.x branch, and despite it being a fairly simple function, it's used in a few core Python modules. So I'm not sure the difficulty of adding it for Python-2.7. Is there an alternative function available in the 2.7 code that I can replace it with? I also don't have a clue how anyone even builds Python-2.7 on Windows, because getting the required tools properly installed borders on the arcane (esp VS2008 tools). So testing it might be a bit difficult. Is there a cheat sheet somewhere? -- nosy: +kumba ___ Python tracker <https://bugs.python.org/issue18597> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
New submission from Joshua Kinard : I'm attempting to get Python to cross-compile, and I'm not sure if this is an actual flaw in the build system or not, but thought I'd detail what I can here and seek comment from those in the know. What happens is under a cross-environment setup on a Gentoo installation (using our sys-devel/crossdev and sys-devel/crossdev-wrappers package), when cross-compiling from x86_64-pc-linux-gnu to mipsel-unknown-linux-gnu, the Python build process fails to build several modules/extensions. I believe that part of the problem with building the extensions is on our end, and is a separate item I'll track down myself. But there is one module in particular that looks tied to Python's core that's getting a cross-compile error: _socket. What happens is, somehow, the configure script (or setup.py) is defining HAVE_SYS_PARAM_H, which pulls in sys/param.h -- I think this is normal, but for socketmodule.c, that particular call by the Makefile passes in -I/usr/include along with the other -I calls defining the cross-include directories. The mipsel cross-compiler then references x86_64-specific assembler code within the sys/param.h copy in /usr/include, and fails. Generally, our crossdev-wrappers package sets up the buil environment and overrides a lot of the common variables to use the cross-toolchain. So far, it looks like only socketmodule.c is affected with the rogue -I/usr/include getting pulled in. I haven't had much luck trying to track down just how Python's build system is tied into autotools to see where it's picking up /usr/include from. Already tried patching setup.py some, as well as passing --oldincludedir to configure, but neither approach worked. I'm hoping that this is either a minor bug in the build system, or we're missing a specific variable to be passed to the make process so that -I/usr/include doesn't get defined. Not sure which, so if there's any other ideas to try, I'm all ears! -- components: Build messages: 83008 nosy: kumba severity: normal status: open title: Cross-compiling Python type: compile error versions: Python 2.5 ___ Python tracker <http://bugs.python.org/issue5404> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
Joshua Kinard added the comment: Gotcha, I'll poke around and see what I can find. Are you guys open to patches for 2.5.x still if we find something that needs patching (versus passing lots of variables to the make process)? ___ Python tracker <http://bugs.python.org/issue5404> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
Joshua Kinard added the comment: Gotcha. Not sure how far off Gentoo is from supporting 2.6 -- our primary package manager relies on it, so the updates tend to be slow. for moving to new versions. Do you guys maintain any kind of an "internals" guide to the build system anywhere? Like an outline or such of what happens from start to finish when you run setup.py (I think that's the start, anyways)? I see there's quite a bit of autotools components plugged in, but it's intermixed with some of the python code itself. Looking at some of the other bugs, there definitely seems to be some cross-compiling capability that's made it in -- as evidenced by a good chunk of the core build cross-compiling fine (the _Socket extension is the one glaring error I'm after at the moment). So it looks like it just needs some better touch upon 2.5 at least. Currently, we're using the patch from Issue #1115, and I'm going to take a stab at adapting the 2.5.1 patch in Issue #1597850 to see if it takes me any farther. ___ Python tracker <http://bugs.python.org/issue5404> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1597850] Cross compiling patches for MINGW
Joshua Kinard added the comment: Anyone gotten farther on getting Python-2.5.x to cross-compile? I'm trying to get x86_64-pc-linux-gnu --> mipsel-unknown-linux-gnu, and after some hacking at the last updated cross-2.5.1.patch, plus a fix for the %zd printf bugaboo, plus adding in config.sub/config.guess files, I'm able to get it moving a little. But I'm running into the same failure as described in Message #56846 and I'm not quite sure how to properly work around that. Can't "hack" around it -- the entire build is automated from a Gentoo ebuild, so I need to be able to tweak something in Makefile.pre.in or configure.in to fix this...somehow. Ran some quick checks, and even with passing -I/usr/include to CPPFLAGS_FOR_BUILD, and running the host compiler directly on the command line, It's picking up wrong values for LONG_BIT and SIZEOF_LONG (I think). LONG_BIT = 64 SIZEOF_LONG = 4 So the test LONG_BIT != (SIZEOF_LONG * 8) succeeds and hits the #error in pyport.h Can't use a newer Python, including 2.6, 2.7-trunk, or even 3.0. Gentoo's primary package manager, portage, is currently dependent on 2.5.x (we're using 2.5.4 specifically right now). So Roumen's patch doesn't work at all (I've already tried backporting). Ideas perhaps? -- nosy: +kumba ___ Python tracker <http://bugs.python.org/issue1597850> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
Joshua Kinard added the comment: Making progress! Adapted the cross-2.5.1.patch from Issue #1597850, integrated the %zd printf fixup patch, and added another cross-compiler check for the libffi configure bits in setup.py (it'd pass libffi's configure no --host options, so libffi would compile for the build system, and that's not good). Now I'm just trying to figure out why for each extension module, why CFLAGS gets passed, as far as I can tell, to both the CC and the LD tools? I see in the code that distutils doesn't allow one to separate CC and CXX. Is the same for CC and LD? binutils' ld doesn't understand the same range of flags that gcc does, so for a mips cross-compile case, you may have the -mabi flag passed. Gcc understands this, but -m on ld is to select the output binary, of which 'abi' isn't (obviously) a valid output type. Is this to me a distutils limitation I'm hitting? I've already dug through the generated Makefile at Python's root and can't pin down precisely where it invokes ld for each extension, so I'm assuming setup.py handles this. However, I don't read python 100%, and some of the tricks you can do with classes and def's seems weird to me. PyBuildExt seems like some of its internal functions call themselves in a recursive fashion -- right? I'm also not sure which function is the actual function call that executes the compiler or the linker. ___ Python tracker <http://bugs.python.org/issue5404> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5404] Cross-compiling Python
Joshua Kinard added the comment:
Roumen,
I took a look at 4010, and tried your patch (as well as attempted to
apply the latter patch, but they changes are too great). Neither one
helped resolve my issue, but I should probably explain the angle I'm
attacking this from so you have an idea of what I'm doing.
Currently, in Gentoo, we have a package file (called an ebuild) for
2.5.4, and this is what our primary package manager, Portage, links up
against, so that's why I'm working on getting 2.5.4 to cross-compile
properly. I have no idea when we will move towards using 2.6.0. I'm
trying to help our embedded team get cross-compiling working in some
packages so we can cross-compile a "base system" from a fast host. This
all started after I decided to resurrect an old Cobalt RaQ2 from the
grave. That machine is extremely limited -- ~144MB of RAM and a 250MHz
RM5231 MIPS Processor with no L2 Cache and a small L1 cache. Compiling
is very, very slow, so the ability to cross-compile about ~83 base
packages in about 2 hours versus an entire day is a very desirable
thing. It has a CHOST tuple of mipsel-unknown-linux-gnu.
Right now, the three most difficult packages to cross-compile are Perl,
gettext, and Python, in that order. Perl is probably beyond all hope.
I don't even know if I'll tackle it. Gettext is just a nightmare, even
for a GNU package. We bypass this by disabling NLS support for the
cross-build. Python, I feel, is doable. It's got some autotools links,
so it's got the capability. Python code is also easy to read for me
(unlike Perl, but I digress), so I can sort of follow the logic in some
spots in setup.py.
We have two packages that assist in cross-compiling. The first is our
'crossdev' package. It is akin to what you'll find in Debian or via Dan
Kegel's crosstools setup. It takes care of installing cross-toolchains
of varying degrees (C-Only, Glibc/Uclibc-based, etc..) on a Gentoo
system. Next, is 'crossdev-wrappers', which is a set of wrapper scripts
for the 'emerge' command, Gentoo's main package manager tool in Portage.
This helps to pass a lot of the basic environment variables so that
common configure scripts can correctly identify the build system and
target/host system types, or allow simple, Makefile-only packages build
using cross-compile toolchains.
So what happens on a cross-compile of Python, is first, the _socket
module fails to cross-compile because for that very specific compiler
invocation, it passes -I/usr/include, and _socket references the
sys/param.h include. Which it finds in my host system's /usr/include
(an amd64 machine currently, so x86_64-pc-linux-gnu is the CHOST tuple).
This will pull in x86 assembler language, which is misunderstood by a
mipsel-based toolchain. I dug around, and found some patches that
re-ordered how configure and setup.py do things, and after utilizing a
tool in our ebuild system to create config.sub/config.guess files, this
helps to get Python's core to compile for the most part (I also applied
a fix for the printf %zd issue).
However, the default compiling logic for python uses gcc to both compile
the *.o files AND link into the final *.so modules. One of the patches
I dug up (and merged) converts this logic to explicitly calling $CC for
compiling the *.o files, and then $LD to link them into *.so modules.
This is where part of my problem lies, because the patch doesn't address
the fact that $CFLAGS gets passed to the $CC call (which is fine) AND to
the $LD call (which is not fine, because 'ld' takes a different set of
arguments) -- via (I think) the $OPT variable.
The other problem was libffi, under the _ctypes/ folder. The setup.py
script explicitly clears configure arguments out via 'config_args = []'
code, and then invokes libffi's configure. Without a --host=${CHOST}
value passed to it, libffi's configure incorrectly assumes it's running
on an x86_64-pc-linux-gnu system, not in a cross-compile environment for
mipsel-unknown-linux-gnu. Thus, the configure script will go on to
build for that system, BUT call the mipsel-unknown-linux-gnu-gcc
compiler to compile its internal test code. This fails, and Python's
default behavior is to ignore build failures and continue on.
So, after addressing those two problems, I've been trying to figure out
how to pass $LD a minimal set of LDFLAGS -- mostly -L commands, although
I'd like to just send it nothing but the *.o files and see how it
handles those by itself. The logic in the makefile is really quite
weird, though, and it's not helped by the patches I'm mixing in, some of
which are a mix of several patches from this bug tracker.
If you want to take a look at the two main patches, I've uploaded them
to an external host, because I didn't want to litter the bug tracker
with incomplete patch
[issue4305] ctypes fails to build on mipsel-linux-gnu (detects mips instead of mipsel)
Joshua Kinard added the comment: Is there any movement on this perchance? Just bumped into this on my MIPS platform and discovered this bug. -- nosy: +kumba ___ Python tracker <http://bugs.python.org/issue4305> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27647] Update Windows build to Tcl/Tk 8.6.6
Joshua Kinard added the comment: Hi, I came across a StackOverflow question that apparently turned up a bug in Tk itself with regards to Spinboxes on Windows Aero themes. The question is here: http://stackoverflow.com/q/30783603/ And I did a deep-dive of the issue here as the accepted answer: http://stackoverflow.com/a/36393680/ This led to someone from Tk noticing, and they patched the bug in Tk upstream in this commit: http://core.tcl.tk/tk/info/f91b7065bf1bf655 I do not know if that commit is in a current Tk release (doubtful). Could you guys look at picking it up for the next Python 2.7/3.x release? It only affects Windows platforms using Aero. Classic mode spinboxes or other Tk themes (e.g., clam) are unaffected. It'd also be awesome if Tk-8.6 could go into the next Python 2.7 release under Windows. That has PNG support for Tk's PhotoImage class, which would make working with icons and transparency a *lot* easier. -- nosy: +kumba ___ Python tracker <http://bugs.python.org/issue27647> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27647] Update Windows build to Tcl/Tk 8.6.6
Joshua Kinard added the comment: @Terry: This only applies on the Windows release of Python. On both Linux and FreeBSD, you can wind up with Python 2.7 and Tk-8.6.x by default in some instances: # pkg info | grep python27 python27-2.7.13_1 Interpreted object-oriented programming language # pkg info | grep tk86 tk86-8.6.6 Graphical toolkit for Tcl That's a FreeBSD-11.0-RELEASE install on a laptop. Is there guidance anywhere on which Tk version a given Python version prefers to be used with? E.g., is Python 2.7 only supported against Tk-8.5.x, leaving 8.6.x users on their own? -- ___ Python tracker <http://bugs.python.org/issue27647> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27647] Update Windows 2.7 build to Tcl/Tk 8.5.19
Joshua Kinard added the comment: Fair enough. I am stuck working with 2.7 for quite a while longer due to platform constraints, but hopefully can migrate over to 3.x at some point. Was worth asking :) At least try to cherrypick that one commit from the TCL/Tk source I referenced so that the Spinbox works correctly under Vista/Aero. I am not sure when that might wind up in a future Tk release. -- ___ Python tracker <http://bugs.python.org/issue27647> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
