[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: Gladly. :-) -- resolution: -> out of date stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___ ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

STINNER Victor added the comment: The PEP 393 has been accepted. There is no more narrow or wide build, Py_UNICODE is now always wchar_t. We have also a stable ABI which doesn't depend on the internal stucture of Unicode strings. In Python 3.2, the name of dynamic libraries is also different

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Éric Araujo : -- nosy: +eric.araujo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Alexander Belopolsky : -- nosy: +belopolsky ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: Thanks, Antoine. Those are all good suggestions. Attached is an updated patch. Marc-Andre, Martin: would you like to look this over as well? -- Added file: http://bugs.python.org/file18730/unicode-3.patch ___ Py

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Antoine Pitrou added the comment: +#undef HAVE_USABLE_CHAR_T Isn't it HAVE_USABLE_WCHAR_T? +struct PY_UNICODE_STRUCT; +typedef struct PY_UNICODE_STRUCT Py_UNICODE_STRUCT; +#define Py_UNICODE Py_UNICODE_STRUCT Why not simply `typedef struct PY_UNICODE_STRUCT Py_UNICODE`? The _testunicodeagnos

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: I just noticed that the previous "new" patch I uploaded was actually my original patch. Oops. Anyway... In a discussion on the capi list, a few people objected to compiling in Unicode-agnostic mode by default (e.g., it would be annoying for Cython). Att

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: Here is a new patch. Usage, short version: By default, modules compile in Unicode-agnostic mode, where Py_UNICODE is an incomplete type. Unicode-agnostic modules will load in both UCS2 and UCS4 interpreters. If a module defines PY_REAL_PY_UNICODE before

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Daniel Stutzbach : Removed file: http://bugs.python.org/file17248/unicode.patch ___ Python tracker ___ ___ Python-bugs-list mailing

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Martin v. Löwis added the comment: > I just thought of another risk inherit in this approach. If the extension > module is composed of more than one C file, the extension author may > inadvertently compile the file defining the PyModuleDef in Unicode-agnostic > mode but compile another file in

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Daniel Stutzbach : Removed file: http://bugs.python.org/file17274/unnamed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Daniel Stutzbach : Removed file: http://bugs.python.org/file17273/unnamed ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sun, May 9, 2010 at 10:12 AM, Marc-Andre Lemburg wrote: > I'm just not sure how you could check for optimization > compiler bugs/features using the buildbots. > Once I have a patch that I'm modestly confident in, I'll write automated tests to go with it.

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 8:07 AM, Martin v. Löwis wrote: > 1. add a flag to PyModuleDef, indicating whether the module was built in > UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module, > instead of having the dynamic linker do so. I jus

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Marc-Andre Lemburg added the comment: Daniel Stutzbach wrote: > > Daniel Stutzbach added the comment: > > On Sat, May 8, 2010 at 11:35 AM, Marc-Andre Lemburg > wrote: >> One of the more important cases you are missing is the >> argument parser in Python: > > Thanks. I've had my head buried

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Martin v. Löwis added the comment: > In Unicode-agnostic mode, instead of leaving Py_UNICODE, > PyUnicodeObject, and many functions undefined, I wonder if it would > be sufficient to declare Py_UNICODE like this: Sounds good. -- ___ Python tracker

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: In Unicode-agnostic mode, instead of leaving Py_UNICODE, PyUnicodeObject, and many functions undefined, I wonder if it would be sufficient to declare Py_UNICODE like this: struct PY_UNICODE_TYPE; typedef struct PY_UNICODE_TYPE Py_UNICODE; That would allow

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 11:35 AM, Marc-Andre Lemburg wrote: > One of the more important cases you are missing is the > argument parser in Python: Thanks. I've had my head buried in c-api/unicode.html and unicodeobject.h. > Py_UNICODE *x; > Py_ssize_t y; > P

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Marc-Andre Lemburg added the comment: Daniel Stutzbach wrote: > > Daniel Stutzbach added the comment: > > On Sat, May 8, 2010 at 10:16 AM, Marc-Andre Lemburg > wrote: >> Are you sure this doesn't get optimized away in practice ? > > I'm sure it doesn't get optimized away by gcc 4.3, where I

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 11:20 AM, Martin v. Löwis wrote: > I'd rather modify PyModuleDef_Base, to include a flags field, and > perhaps put the api version into it, as well. That's an ABI change, > unfortunately, but such a flags field may turn out useful, anyw

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Martin v. Löwis added the comment: > In most cases that will Just Work, without requiring the module writer > to modify their PyModuleDef. I'd rather modify PyModuleDef_Base, to include a flags field, and perhaps put the api version into it, as well. That's an ABI change, unfortunately, but suc

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 11:02 AM, Martin v. Löwis wrote: > Did you mean to include the hunk from msg105295 as part of the patch? > If so, wouldn't that defeat the whole point of the patch? My intention is to offer two compilation modes to extension authors:

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 8:07 AM, Martin v. Löwis wrote: > 1. add a flag to PyModuleDef, indicating whether the module was built in > UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module, > instead of having the dynamic linker do so. > 2.

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Martin v. Löwis added the comment: >> Are you sure this doesn't get optimized away in practice ? > > I'm sure it doesn't get optimized away by gcc 4.3, where I tested it. :) Did you mean to include the hunk from msg105295 as part of the patch? If so, wouldn't that defeat the whole point of the

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 10:16 AM, Marc-Andre Lemburg wrote: > Are you sure this doesn't get optimized away in practice ? I'm sure it doesn't get optimized away by gcc 4.3, where I tested it. :) > Sure, though, I don't see how this relates to C code relying >

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Marc-Andre Lemburg added the comment: Daniel Stutzbach wrote: > > Daniel Stutzbach added the comment: > > On Sat, May 8, 2010 at 5:03 AM, Marc-Andre Lemburg > wrote: >> If you can propose a different method of reliably protecting against >> mixed Unicode build module loads, that would be gre

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: On Sat, May 8, 2010 at 5:03 AM, Marc-Andre Lemburg wrote: > If you can propose a different method of reliably protecting against > mixed Unicode build module loads, that would be great. We could then > get rid off the wrapping altogether. The following code

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Marc-Andre Lemburg added the comment: Martin v. Löwis wrote: > > Martin v. Löwis added the comment: > > I propose a different approach: > > 1. add a flag to PyModuleDef, indicating whether the module was built in > UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module, >

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Martin v. Löwis added the comment: I propose a different approach: 1. add a flag to PyModuleDef, indicating whether the module was built in UCS-2 or UCS-4 mode. Then let the interpreter refuse the load the module, instead of having the dynamic linker do so. 2. provide a mode for the header fi

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Marc-Andre Lemburg added the comment: Daniel Stutzbach wrote: > > New submission from Daniel Stutzbach : > > Currently, Python can be built with an internal Unicode representation of > UCS2 or UCS4. To prevent extension modules compiled with the wrong Unicode > representation from linking,

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Zooko O'Whielacronx added the comment: > Option 1: Make Unicode-agnosticism the default and force anyone who cares > about the Unicode setting to include a separate header. If they don't > include that header, they can only call safe functions and can't poke at > PyUnicodeObject's internals.

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

R. David Murray added the comment: Adding MvL because he wrote the ABI PEP, and MAL because he cares about the Unicode interface. -- nosy: +lemburg, loewis, r.david.murray ___ Python tracker __

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: I've been thinking about this a bit more. There are three types of symbols in unicodeobject.h: 1. Functions that are always safe to use 2. Functions that are only safe if the module is compiled with the same Unicode settings as Python 3. Structures and mac

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: Adding people to the Nosy List who participated in the original thread on python-ideas, several months ago. Hope you don't mind. :-) -- nosy: +gvanrossum, scoder, zooko ___ Python tracker

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Daniel Stutzbach added the comment: Forcing the compile-time and link-time encodings to match is tricky. The goal is: - Allow Unicode-agnostic modules to link, regardless of unicode settings - Cause a link failure if the unicode settings are mismatched for a module that pokes into PyUnicodeO

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

Changes by Daniel Stutzbach : -- keywords: +patch Added file: http://bugs.python.org/file17248/unicode.patch ___ Python tracker ___ ___

[issue8654] Improve ABI compatibility between UCS2 and UCS4 builds

New submission from Daniel Stutzbach : Currently, Python can be built with an internal Unicode representation of UCS2 or UCS4. To prevent extension modules compiled with the wrong Unicode representation from linking, unicodeobject.h #defines many of the Unicode functions. For example, PyUnic