[Python-Dev] Constifying C API
Originally C API didn't use the const qualifier. Over few last years the const qualifier was added to C API if that preserved backward compatibility. For example input "char *" parameters were changed to "const char *". This makes C API compatible with C++, eliminates C compiler warnings, and helps to found possible errors. Now I have started to make changes that are not absolute compatible, and can need modifying third-party code (but unlikely). * The const qualifier was added to "char *" fields name and doc of some structures. They always point to C string literals. https://bugs.python.org/issue28761 * The const qualifier was added to private global variable _Py_PackageContext. https://bugs.python.org/issue28748 Now I'm going to add the const qualifier to the result of PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8(). These functions return a reference to internal cached UTF8 representations of a string. It should never be modified. https://bugs.python.org/issue28769 Later I'm planning following changes: * Add the const qualifier to the result of functions that return references to internal representation of immutable objects, like PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally can modify the content of immutable objets, this is very dangerous, because this can invalidates invariants and cached values. Third-party code shouldn't do this. * Add the const qualifier to the format field of Py_buffer. It is a reference to C string literal or to the content of bytes object. Mutating its content is an error. Only _testbuffer overuses the format field of internal Py_buffer object for owning a reference to allocated memory. But this is not leaked outside. What are you think about this? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Constifying C API
On 18 December 2016 at 18:31, Serhiy Storchaka wrote: > Later I'm planning following changes: > > * Add the const qualifier to the result of functions that return > references to internal representation of immutable objects, like > PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally can > modify the content of immutable objets, this is very dangerous, because > this can invalidates invariants and cached values. Third-party code > shouldn't do this. > > * Add the const qualifier to the format field of Py_buffer. It is a > reference to C string literal or to the content of bytes object. Mutating > its content is an error. Only _testbuffer overuses the format field of > internal Py_buffer object for owning a reference to allocated memory. But > this is not leaked outside. > > What are you think about this? > As long as it's on the default branch with appropriate notes in the C porting section of the 3.7 What's New, turning these kinds of runtime errors into compilation errors sounds like the right thing to do to me. One key aspect from my perspective is that code that is updated to correctly declare the destination storage as a const pointer will still compile against the old API variants that return a mutable pointer, so any problems this finds in third party code are likely to be resolved for older 3.x releases as well. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Constifying C API
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/18/2016 07:54 PM, Nick Coghlan wrote: > On 18 December 2016 at 18:31, Serhiy Storchaka > wrote: > >> Later I'm planning following changes: >> >> * Add the const qualifier to the result of functions that return >> references to internal representation of immutable objects, like >> PyBytes_AS_STRING() or PyUnicode_DATA(). While CPython internally >> can modify the content of immutable objets, this is very dangerous, >> because this can invalidates invariants and cached values. >> Third-party code shouldn't do this. >> >> * Add the const qualifier to the format field of Py_buffer. It is a >> reference to C string literal or to the content of bytes object. >> Mutating its content is an error. Only _testbuffer overuses the >> format field of internal Py_buffer object for owning a reference to >> allocated memory. But this is not leaked outside. >> >> What are you think about this? >> > > As long as it's on the default branch with appropriate notes in the C > porting section of the 3.7 What's New, turning these kinds of runtime > errors into compilation errors sounds like the right thing to do to > me. > > One key aspect from my perspective is that code that is updated to > correctly declare the destination storage as a const pointer will > still compile against the old API variants that return a mutable > pointer, so any problems this finds in third party code are likely to > be resolved for older 3.x releases as well. Agreed. Anything the compiler ralfs on after adding 'const' (where the actual target must be immutable) already had the fuse smoldering. FWIW I help maintain some *old* C extensions (fifteen+ years and counting), and am as likely to be affected as anyone. Tres - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJYVz1oAAoJEPKpaDSJE9HYNc0P/2ZfDQeWmecy/deL4mqvLh42 iZuyyXoYmsEvHWgTL1gCOK3isUAKn5MMDAk79ezGkbmrmerxV0EVCrIsQMaCBuhY ypWxsPHa1nUpJpTuziHi452ETDq606nDgUXJnNUtR1xqVlFNpNskTYdexkxv4K5W E+ANwNvE+/YZN7t8KmIcR8pczRGhWJ5X67+etG0KlJ0mDR13RIpUZs7OfTFsXRi1 YHYgI1uKKkphB/KdPxeQfN4G5CgiRK3fJ8sQO2ojJKt3xMqPJcmGG0KIHZi0waXA Uqh+ukKE1tWDdBPYubv+4nlrtWQye6kX9gUu/gXYXM9C7h3u9B9otYXblNGqZAol q6+QfnSmOCZkGeaGw+Gwzz+B2yQcz4phuaz1AirtYUA66s0vbLuKi+SNiVei2gzn M/xd1HpZOxFVk/QkZYHlOW0k2F8o73ecWSONo1xTgi7pdjDrAALhbQ+7Z/dBHn0i 474VoRXcEqVwST87CqbEXyW82GexOppPGqi0jgeAFWJtb0HytuLv21l/h7XgX/TV lmrxGAh6VGl2FOIQolgSNKaVQHsxh2xDq8lL7hGgXuDcI4fD3d+p6bu3tpN6nXMA b4k0TAry7PfKASk0MJgU9aZCSFulDR8ghnx+nUte0OrDdd+nqaovtZcT1Y52glU/ FBw00PcU9+MWZ+zlQNfs =/M++ -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Should I delay 3.5.3 and 3.4.6 by two weeks?
Python 3.6.0 final just slipped by two weeks. I scheduled 3.5.3 and 3.4.6 to ship about a month after 3.6.0 did, to "let the dust settle" around the release. I expect a flood of adoption of 3.6, and people switching will find bugs, and maybe those bugs are in 3.5 or 3.4. So it just seemed sensible. 3.6 just slipped by two weeks. So now there's less than two weeks between 3.6.0 final shipping and tagging the release canddiates for 3.5.3 and 3.4.6. This isn't as much time as I'd like. If I had total freedom to do as I liked, I'd slip my releases by two weeks to match 3.6. But there might be people planning around 3.5.3 and 3.4.6--like Guido was waiting for 3.5.3 for something iirc. So, if you have an opinion, please vote for one of these three options: * Don't slip 3.5.3. and 3.4.6. * Slip 3.5.3 and 3.4.6 by two weeks to match 3.6.0. * Slip 3.5.3 and 3.4.6 by a whole month, to give 3.6.0 the ability to slip again without us having to change the release. Your faithful servant, //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should I delay 3.5.3 and 3.4.6 by two weeks?
On Dec 19, 2016, at 00:26, Larry Hastings wrote: > Python 3.6.0 final just slipped by two weeks. While it should not affect decisions about 3.5.3 and 3.4.6, so there's no confusion: the 3.6.0 release date slipped one week, from 2016-12-16 to 2016-12-23. Of course, until the release happens, it's possible that it could slip again but it hasn't yet and we are going to do our best to keep it from doing so. -- Ned Deily [email protected] -- [] ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
