Re: [Python-Dev] libpython added to ABI tracker
On 1 April 2014 21:23, Andrey Ponomarenko wrote: > > Nick Coghlan wrote: >> >> On 1 Apr 2014 01:38, "Victor Stinner" wrote: >>> >>> 2014-03-31 13:38 GMT+02:00 Andrey Ponomarenko : The public libpython API changes will be tracked here: http://upstream-tracker.org/versions/python_public_api.html For now I've excluded only symbols starting with an underscore. What >> >> other symbols should be excluded? >>> >>> It's not a matter of underscore. You should define Py_LIMITED_API to >>> 0x0302 to test the stable ABI of Python 3.2. >>> >>> http://docs.python.org/dev/c-api/stable.html >> >> Well, we have more than one ABI, with different guarantees. The "no >> leading >> underscore" one we promise not to change in maintenance releases, but we >> only preserve *API* compatibility in feature releases (mostly due to >> structs changing size). >> >> The "stable ABI" (aka Py_LIMITED_API) is the one where we promise to hide >> all the memory layout details and treat it as "additive only" so that >> binaries built with previous releases keep working. That should never >> break >> ABI compatibility, and only get new additions if the macro definition is >> bumped up to match the newer release. >> >> Cheers, >> Nick. > > > The stable libpython ABI with Py_LIMITED_API=0x0302 will be tracked at > http://upstream-tracker.org/versions/python_stable_api.html Thanks! The "leading underscore means private" convention is also applicable here (that's a general guideline for Python related APIs). Interesting to see the UCS2 removal there for 3.3. That's a genuine removal from the public ABI as part of PEP 393. I guess the reason nobody complained is because most 3.2 Linux builds used the UCS4 ABI instead, and the stable ABI hadn't seen broad adoption on Windows in the 3.2->3.3 time frame. Regarding the warnings for this one - is there a way for the checker to warn if data structures are exposed directly, rather than as opaque types? It's fine if there isn't, it would just be cool if there was - one of the premises of the stable ABI is that it *doesn't* expose the type definitions directly to consuming code, just the pointers to them (hence allowing the struct size to change without actually breaking compatibility with the defined ABI). Regardless, this service already shows we've made some mistakes with the stable ABI in previous releases - it is indicating there are new symbols in the stable ABI for 3.3 and 3.4 that aren't properly guarded with version constraints. That means it is currently possible to set Py_LIMITED_API=0x0302 and get something that won't actually run properly on 3.2. Georg, Larry, Benjamin - should checking these be added to PEP 101, so we don't get the same thing happening for 3.5? > I also added source-compatibility reports to the "public" API tracker: > http://upstream-tracker.org/versions/python_public_api.html Thanks again for setting these up! Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] libpython added to ABI tracker
On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote: > Georg, Larry, Benjamin - should checking these be added to PEP 101, so > we don't get the same thing happening for 3.5? I would like it if we could put this in the testsuite somehow. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] libpython added to ABI tracker
Nick Coghlan wrote: On 1 Apr 2014 01:38, "Victor Stinner" wrote: 2014-03-31 13:38 GMT+02:00 Andrey Ponomarenko : The public libpython API changes will be tracked here: http://upstream-tracker.org/versions/python_public_api.html For now I've excluded only symbols starting with an underscore. What other symbols should be excluded? It's not a matter of underscore. You should define Py_LIMITED_API to 0x0302 to test the stable ABI of Python 3.2. http://docs.python.org/dev/c-api/stable.html Well, we have more than one ABI, with different guarantees. The "no leading underscore" one we promise not to change in maintenance releases, but we only preserve *API* compatibility in feature releases (mostly due to structs changing size). The "stable ABI" (aka Py_LIMITED_API) is the one where we promise to hide all the memory layout details and treat it as "additive only" so that binaries built with previous releases keep working. That should never break ABI compatibility, and only get new additions if the macro definition is bumped up to match the newer release. Cheers, Nick. P.S. I understand it was Anatoly that put the process in motion to get this set up. Thanks for doing that Anatoly, it's a genuinely good idea. The stable libpython ABI with Py_LIMITED_API=0x0302 will be tracked at http://upstream-tracker.org/versions/python_stable_api.html I also added source-compatibility reports to the "public" API tracker: http://upstream-tracker.org/versions/python_public_api.html Thanks. -- Andrey Ponomarenko, NTC IT ROSA. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (3.4): simplify check, since now there are only new-style classes
On Tue, Apr 1, 2014 at 1:21 PM, benjamin.peterson wrote: > http://hg.python.org/cpython/rev/abb85902ce79 > changeset: 90090:abb85902ce79 > branch: 3.4 > parent: 90088:4a2dabac976d > user:Benjamin Peterson > date:Tue Apr 01 14:20:56 2014 -0400 > summary: > simplify check, since now there are only new-style classes > > files: > Lib/urllib/request.py | 7 ++- > 1 files changed, 2 insertions(+), 5 deletions(-) > > > diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py > --- a/Lib/urllib/request.py > +++ b/Lib/urllib/request.py > @@ -511,9 +511,6 @@ > If any of the handlers passed as arguments are subclasses of the > default handlers, the default handlers will not be used. > """ > -def isclass(obj): > -return isinstance(obj, type) or hasattr(obj, "__bases__") > - > opener = OpenerDirector() > default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, > HTTPDefaultErrorHandler, HTTPRedirectHandler, > @@ -524,7 +521,7 @@ > skip = set() > for klass in default_classes: > for check in handlers: > -if isclass(check): > +if instance(check, type): Should be isinstance, should it not? -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pickle self-delimiting
Hi, Unless I'm mistaken, pickle's documentation doesn't mention that the pickle wire-format is self-delimiting. Is there any reason why it's not documented? The reason I'm asking is because I've seen some code out there doing its own ad-hoc length-prefix framing. Cheers, cf ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.4): simplify check, since now there are only new-style classes
On Tue, 1 Apr 2014 13:28:53 -0500 Zachary Ware wrote: > > @@ -524,7 +521,7 @@ > > skip = set() > > for klass in default_classes: > > for check in handlers: > > -if isclass(check): > > +if instance(check, type): > > Should be isinstance, should it not? Sounds like a well-tested code path :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pickle self-delimiting
On Tue, 1 Apr 2014 19:29:38 +0100 Charles-François Natali wrote: > Hi, > > Unless I'm mistaken, pickle's documentation doesn't mention that the pickle > wire-format is self-delimiting. Is there any reason why it's not documented? No reason AFAIK. However, the fact that it is self-delimited is implicit in the fact that "Bytes past the pickled object’s representation are ignored": https://docs.python.org/dev/library/pickle.html#pickle.load Also, note that protocol 4 now features a length-prefix framing to improve buffering performance with arbitrary streams. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] libpython added to ABI tracker
On 2 Apr 2014 00:54, "Benjamin Peterson" wrote: > > On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote: > > Georg, Larry, Benjamin - should checking these be added to PEP 101, so > > we don't get the same thing happening for 3.5? > > I would like it if we could put this in the testsuite somehow. Perhaps an independent automated daily or weekly check like Antoine's daily refleak hunter would be better? Cheers, Nick. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] libpython added to ABI tracker
On Tue, Apr 1, 2014, at 13:59, Nick Coghlan wrote: > On 2 Apr 2014 00:54, "Benjamin Peterson" wrote: > > > > On Tue, Apr 1, 2014, at 4:45, Nick Coghlan wrote: > > > Georg, Larry, Benjamin - should checking these be added to PEP 101, so > > > we don't get the same thing happening for 3.5? > > > > I would like it if we could put this in the testsuite somehow. > > Perhaps an independent automated daily or weekly check like Antoine's > daily > refleak hunter would be better? As long as it fails loudly. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pickle self-delimiting
> No reason AFAIK. However, the fact that it is self-delimited is implicit > in the fact that "Bytes past the pickled object's representation are > ignored": https://docs.python.org/dev/library/pickle.html#pickle.load I find this sentence worrying: it could lead one to think that load() could read more bytes than the expected object representation size: this would make pickle actually non self-delimiting, and could lead to problems when reading e.g. from a socket, since an extraneous read() could block. I think it's worth making it clear in the doc, I'll open an issue on the tracker. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
