[Python-Dev] PEP 384 status
I have now started an initial patch for PEP 384, in the pep-0384 branch. This has the following features: - modules can be compiled under Py_LIMITED_API - Tools/scripts/abitype.py converts C code containing static PyTypeObject definitions to use the new API for type definitions. The following aspects are still missing: - there is no support for generating python3.dll on Windows yet - there has been no validation whether the API is actually feasible to use in extension modules. I started looking into porting the sqlite extension, and ran into these issues: - certain fields of PyTypeObject are called directly: pysqlite_NodeType.tp_alloc Py_TYPE(self)->tp_free - PyObject_Print is used, but can't be supported, as it uses a FILE* parameter For the first issue, it would be possible to provide a generic accessor function that fetches fields from a type object. Alternatively, each case could be considered, suggesting an alternative code for the desired effect. I'll be off the net for the next two weeks most of the time, so I might not be able to respond quickly. Anybody interested in advancing that patch, feel free to commit changes into the branch. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] versioned .so files for Python 3.2
>> This leads me to a question: how do these configure options affect the >> PEP 384 stable ABI? That PEP is currently silent on the issue, while >> PEP 3149 appears to implicitly assume that "abi3" completely specifies >> the ABI. > > It's a great question - perhaps Martin can chime in? It may be that 'abiX' > isn't enough to fully specify compatible extension modules even when that > module is written entirely and solely against PEP 384. In that case, we may > need to include the configure flags in the tag, e.g. foo.abi3-dmu.so. The intention is that there is indeed just one stable ABI, so one configuration is the supported one, and that should be the "default" build. As for the specific settings, my analysis would be this: - pydebug: not supported by the stable ABI, as it changes the layout of PyObject, which is an exposed structure More specifically: Py_DEBUG, Py_TRACEREFS and Py_REF_DEBUG are all incompatible with the stable ABI - pymalloc: I fail to see the impact on the ABI. All allocator macros become function calls under Py_LIMITED_API, otherwise, there shouldn't be any need to have different versions of that. - wide-unicode: this is a tricky one. I'm tempted to say that the stable ABI should always use a Py_UNICODE that matches the platform's wchar_t. Alternative proposals are welcome. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
2010/8/28 "Martin v. Löwis" : > I have now started an initial patch for PEP 384, in the pep-0384 branch. > This has the following features: > - modules can be compiled under Py_LIMITED_API > - Tools/scripts/abitype.py converts C code containing static > PyTypeObject definitions to use the new API for type definitions. > > The following aspects are still missing: > - there is no support for generating python3.dll on Windows yet > - there has been no validation whether the API is actually feasible > to use in extension modules. > > I started looking into porting the sqlite extension, and ran into > these issues: > - certain fields of PyTypeObject are called directly: > pysqlite_NodeType.tp_alloc > Py_TYPE(self)->tp_free This is from tp_new and tp_dealloc, right? I think we should probably provide assessors PyObject_Alloc and PyObject_FreeObject. > - PyObject_Print is used, but can't be supported, as it uses a FILE* > parameter I thought tp_print was supposed to have been removed. Anyway, if sqlite is already using FILE *, then won't it be afflicted by the Microsoft runtime version changes anyway? Maybe provide an extra flag to enable FILE* APIs for those extensions that want to risk it? -- Regards, Benjamin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
> This is from tp_new and tp_dealloc, right? I think we should probably > provide assessors PyObject_Alloc and PyObject_FreeObject. Correct, and yes, that sounds like a good approach. >> - PyObject_Print is used, but can't be supported, as it uses a FILE* >> parameter > > I thought tp_print was supposed to have been removed. Yes - that should have happened for 3.0. Not sure how to deal with it now. > Anyway, if > sqlite is already using FILE *, then won't it be afflicted by the > Microsoft runtime version changes anyway? Maybe provide an extra flag > to enable FILE* APIs for those extensions that want to risk it? For the sqlite extension, that wouldn't be a problem: if they build with a different MSVC release, it will automatically link with a different CRT, which then will have a consistent set of FILE objects. The issue only arises if you pass FILE* across DLLs which in turn are linked with different CRTs. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 status
On Sun, Aug 29, 2010 at 4:52 AM, "Martin v. Löwis" wrote:
>> This is from tp_new and tp_dealloc, right? I think we should probably
>> provide assessors PyObject_Alloc and PyObject_FreeObject.
>
> Correct, and yes, that sounds like a good approach.
>
>>> - PyObject_Print is used, but can't be supported, as it uses a FILE*
>>> parameter
>>
>> I thought tp_print was supposed to have been removed.
>
> Yes - that should have happened for 3.0. Not sure how to deal with it now.
tp_print actually is gone, but PyObject_Print was retained. It just
relies on repr() and str() under the hood instead of the old tp_print
slot.
There are 4 operations performed on fp in that function: clearerr,
ferror, fprintf and fwrite.
There is also an implicit reference to errno (through
PyErr_SetFromErrno) which can't be trusted in a mixed CRT world
(PyErr_SetFromErrno() should be excluded from the Limited API, if it
isn't already, and replaced with a PyErr_SetFromErrnoEx which takes
the errno as an explicit argument. That assumes the CRTs will be
sufficiently compatible that strerror() will give the correct answer
for errno values from the other CRT. If that assumption is incorrect,
the new function would also need to accept a string argument for the
error description).
Four options come to mind:
- just leave it out of the limited API, extensions can do their own
thing to print objects
- leave PyObject_Print out of the limited API, but create a
PyObject_PrintEx that takes a Python IO stream via PyObject* rather
than a C level FILE*.
- leave PyObject_Print out of the limited API, but create a
PyObject_PrintEx that takes function pointers for the above 4
operations (so the FILE* pointer is only every operated on by
functions from the extension module's CRT)
- leave PyObject_Print out of the limited API, but create a
PyObject_PRINT macro that does much the same thing with the logic
rearranged so there is an inner function that figures out the string
to be printed, but an outer macro that does all the operations on the
FILE * object (so again, the FILE * is never passed to Python's CRT)
The last option requires the fewest adjustments for extension authors,
and it should be feasible to do it that way (even though it is a bit
of a hack). Something along the lines of the following:
#define PyObject_PRINT (obj, fp, flags, resultp) \
{ \
int _result = -1;
_t = _PyObject_PrintInner(obj, flags); \
if (_t != NULL) { \
clearerr(fp); \
fwrite(PyBytes_AS_STRING(_t), 1, PyBytes_GET_SIZE(_t), fp); \
Py_DECREF(_t); \
if (ferror(fp)) { \
PyErr_SetFromErrnoEx(PyExc_IOError, errno); \
clearerr(fp); \
} else { \
_result = 0; \
} \
} \
if (resultp != NULL) { \
*resultp = _result; \
} \
}
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
