Re: [Python-Dev] __qualname__ exposed as a local variable: standard?
Hi list, > I noticed __qualname__ is exposed by locals() while defining a class. This > is handy but I'm not sure about its status: is it standard or just an > artifact of the current implementation? (btw, the pycodestyle linter -former > pep8- rejects its usage). I was unable to find any reference to this > behavior in PEP 3155 nor in the language reference. I would like to underline the importance of this question, and give some background, as it happens to resonate with my work on PEP 487. The __qualname__ of a class is originally determined by the compiler. One might now think that it would be easiest to simply set the __qualname__ of a class once the class is created, but this is not as easy as it sounds. The class is created by its metaclass, so possibly by user code, which might create whatever it wants, including something which is not even a class. So the decision had been taken to sneak the __qualname__ through user code, and pass it to the metaclasses __new__ method as part of the namespace, where it is deleted from the namespace again. This has weird side effects, as the namespace may be user code as well, leading to the funniest possible abuses, too obscene to publish on a public mailing list. A very different approach has been taken for super(). It has similar problems: the zero argument version of super looks in the surrounding scope for __class__ for the containing class. This does not exist yet at the time of creation of the methods, so a PyCell is put into the function's scope, which will later be filled. It is actually filled with whatever the metaclasses __new__ returns, which may, as already be said, anything (some sanity checks are done to avoid crashing the interpreter). I personally prefer the first way of doing things like for __qualname__, even at the cost of adding things to the classes namespace. It could be moved after the end of the class definition, such that it doesn't show up while the class body is executed. We might also rename it to __@qualname__, this way it cannot be accessed by users in the class body, unless they look into locals(). This has the large advange that super() would work immediately after the class has been defined, i.e. already in the __new__ of the metaclass after it has called type.__new__. All of this changes the behavior of the interpreter, but we are talking about undocumented behavior. The changes necessary to make super() work earlier are store in http://bugs.python.org/issue23722 Greetings Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP487: Simpler customization of class creation
Hi list, another round for PEP 487, is there any chance it still makes it into Python 3.6? The PEP should be effectively done, I updated the examples in it, given that I implemented the PEP I could actually test the examples, so now they work. The implementation is at http://bugs.python.org/issue27366, including documentation and tests. Unfortunately nobody has reviewed the patch yet. The new version of the PEP is attached. Greetings Martin PEP: 487 Title: Simpler customisation of class creation Version: $Revision$ Last-Modified: $Date$ Author: Martin Teichmann , Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Feb-2015 Python-Version: 3.6 Post-History: 27-Feb-2015, 5-Feb-2016, 24-Jun-2016, 2-Jul-2016, 13-Jul-2016 Replaces: 422 Abstract Currently, customising class creation requires the use of a custom metaclass. This custom metaclass then persists for the entire lifecycle of the class, creating the potential for spurious metaclass conflicts. This PEP proposes to instead support a wide range of customisation scenarios through a new ``__init_subclass__`` hook in the class body, and a hook to initialize attributes. The new mechanism should be easier to understand and use than implementing a custom metaclass, and thus should provide a gentler introduction to the full power of Python's metaclass machinery. Background == Metaclasses are a powerful tool to customize class creation. They have, however, the problem that there is no automatic way to combine metaclasses. If one wants to use two metaclasses for a class, a new metaclass combining those two needs to be created, typically manually. This need often occurs as a surprise to a user: inheriting from two base classes coming from two different libraries suddenly raises the necessity to manually create a combined metaclass, where typically one is not interested in those details about the libraries at all. This becomes even worse if one library starts to make use of a metaclass which it has not done before. While the library itself continues to work perfectly, suddenly every code combining those classes with classes from another library fails. Proposal While there are many possible ways to use a metaclass, the vast majority of use cases falls into just three categories: some initialization code running after class creation, the initalization of descriptors and keeping the order in which class attributes were defined. The first two categories can easily be achieved by having simple hooks into the class creation: 1. An ``__init_subclass__`` hook that initializes all subclasses of a given class. 2. upon class creation, a ``__set_owner__`` hook is called on all the attribute (descriptors) defined in the class, and The third category is the topic of another PEP 520. As an example, the first use case looks as follows:: >>> class QuestBase: ...# this is implicitly a @classmethod ...def __init_subclass__(cls, swallow, **kwargs): ...cls.swallow = swallow ...super().__init_subclass__(**kwargs) >>> class Quest(QuestBase, swallow="african"): ...pass >>> Quest.swallow 'african' The base class ``object`` contains an empty ``__init_subclass__`` method which serves as an endpoint for cooperative multiple inheritance. Note that this method has no keyword arguments, meaning that all methods which are more specialized have to process all keyword arguments. This general proposal is not a new idea (it was first suggested for inclusion in the language definition `more than 10 years ago`_, and a similar mechanism has long been supported by `Zope's ExtensionClass`_), but the situation has changed sufficiently in recent years that the idea is worth reconsidering for inclusion. The second part of the proposal adds an ``__set_owner__`` initializer for class attributes, especially if they are descriptors. Descriptors are defined in the body of a class, but they do not know anything about that class, they do not even know the name they are accessed with. They do get to know their owner once ``__get__`` is called, but still they do not know their name. This is unfortunate, for example they cannot put their associated value into their object's ``__dict__`` under their name, since they do not know that name. This problem has been solved many times, and is one of the most important reasons to have a metaclass in a library. While it would be easy to implement such a mechanism using the first part of the proposal, it makes sense to have one solution for this problem for everyone. To give an example of its usage, imagine a descriptor representing weak referenced values:: import weakref class WeakAttribute: def __get__(self, instance, owner): return instance.__dict__[self.name]() def __set__(self, instance, value): instance.__dict__[self.name] = weakref.ref(value) # this is the new initializer:
Re: [Python-Dev] Status of Python 3.6 PEPs?
Hi, > "PEP 487 -- Simpler customisation of class creation" > https://www.python.org/dev/peps/pep-0487/ > => draft I would like to get that into Python 3.6. It's already implemented, including documentation and tests (http://bugs.python.org/issue27366). Greetings Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
On 14 July 2016 at 00:15, Martin Teichmann wrote: > Hi list, > > another round for PEP 487, is there any chance it still makes it into > Python 3.6? > > The PEP should be effectively done, I updated the examples in it, > given that I implemented the PEP I could actually test the examples, > so now they work. > > The implementation is at http://bugs.python.org/issue27366, including > documentation and tests. Unfortunately nobody has reviewed the patch > yet. > > The new version of the PEP is attached. +1 from me for this version - between them, this and PEP 520 address everything I hoped to achieve with PEP 422, and a bit more besides. There's no BDFL delegation in place for this one though, so it's really Guido's +1 that you need :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyPI index deprecation
Hi, as you probably already know, today the PyPI index page ( https://pypi.python.org/pypi?%3Aaction=index) was deprecated and ceased to be. Among other things it affected PyCharm IDE that relied on that page to enable packaging related features from the IDE. As a result users of PyCharm can no longer install/update PyPI packages from PyCharm. Here is an issue about that in our tracker: https://youtrack.jetbrains.com/issue/PY-20081 Given that there are several hundred thouthands of PyCharm users in the world -- all 3 editions: Professional, Community, and Educational are affected -- this can lead to a storm of a negative feedback, when people will start to face the denial of the service. The deprecation of the index was totally unexpected for us and we weren't prepared for that. Maybe we missed some announcement. We will be very happy if the functionality of the index is restored at least for some short period of time: please, give as a couple of weeks. That will allow us to implement a workaround and provide the fix for the several latest major versions of PyChram. Does anybody know who is responsible for that decision and whom to connect about it? Please help. Best regards, Dmitry Trofimov PyCharm Team Lead JetBrainshttp://www.jetbrains.com The Drive To Develop ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI index deprecation
You may know that there are approximately 3 pypi maintainers, all overworked and one paid. It is amazing that it works at all. I don't know anything about that particular decision though. On Wed, Jul 13, 2016 at 1:21 PM Dmitry Trofimov < [email protected]> wrote: > Hi, > > as you probably already know, today the PyPI index page ( > https://pypi.python.org/pypi?%3Aaction=index) was deprecated and ceased > to be. > > Among other things it affected PyCharm IDE that relied on that page to > enable packaging related features from the IDE. As a result users of > PyCharm can no longer install/update PyPI packages from PyCharm. > > Here is an issue about that in our tracker: > https://youtrack.jetbrains.com/issue/PY-20081 > > Given that there are several hundred thouthands of PyCharm users in the > world -- all 3 editions: Professional, Community, and Educational are > affected -- this can lead to a storm of a negative feedback, when people > will start to face the denial of the service. > > The deprecation of the index was totally unexpected for us and we weren't > prepared for that. Maybe we missed some announcement. > > We will be very happy if the functionality of the index is restored at > least for some short > period of time: please, give as a couple of weeks. That will allow us to > implement a workaround and provide the fix for the several latest major > versions of PyChram. > > Does anybody know who is responsible for that decision and whom to > connect about it? Please help. > > Best regards, > > Dmitry Trofimov > PyCharm Team Lead > JetBrainshttp://www.jetbrains.com > The Drive To Develop > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/dholth%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI index deprecation
Dmitry Trofimov jetbrains.com> writes: > > We will be very happy if the functionality of the index is restored at least > for some short period of time: please, give as a couple of weeks. That will > allow us to implement a workaround and provide the fix for the several latest > major versions of PyChram. > We've temporarily restored this url. It would be easiest if you could come into distutils-sig as I'm not subscribed to python-dev (posting from gmane, hopefully this works!) and we can help you figure out how to use the officially supported APIs to access the information you need as well as figure out a date when we can shut it down again? We want to disable this particular URL because it accounts for a large amount of the slow downs in PyPI and we're not going to be able to continue supporting it into the future. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPI index deprecation
Hi Donald, thanks for your immediate response! Let's move the discussion to the distutils-sig. Best regards, Dmitry On Wed, Jul 13, 2016 at 7:43 PM, Donald Stufft wrote: > Dmitry Trofimov jetbrains.com> writes: > > > > > We will be very happy if the functionality of the index is restored at > least > > for some short period of time: please, give as a couple of weeks. That > will > > allow us to implement a workaround and provide the fix for the > several latest > > major versions of PyChram. > > > > We've temporarily restored this url. It would be easiest if you could come > into > distutils-sig as I'm not subscribed to python-dev (posting from gmane, > hopefully this works!) and we can help you figure out how to use the > officially > supported APIs to access the information you need as well as figure out a > date > when we can shut it down again? We want to disable this particular URL > because > it accounts for a large amount of the slow downs in PyPI and we're not > going to > be able to continue supporting it into the future. > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/dmitry.trofimov%40jetbrains.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
I'm reviewing this now. Martin, can you please submit the new version of your PEP as a Pull Request to the new peps repo on GitHub? https://github.com/python/peps --Guido On Wed, Jul 13, 2016 at 7:45 AM, Nick Coghlan wrote: > On 14 July 2016 at 00:15, Martin Teichmann wrote: >> Hi list, >> >> another round for PEP 487, is there any chance it still makes it into >> Python 3.6? >> >> The PEP should be effectively done, I updated the examples in it, >> given that I implemented the PEP I could actually test the examples, >> so now they work. >> >> The implementation is at http://bugs.python.org/issue27366, including >> documentation and tests. Unfortunately nobody has reviewed the patch >> yet. >> >> The new version of the PEP is attached. > > +1 from me for this version - between them, this and PEP 520 address > everything I hoped to achieve with PEP 422, and a bit more besides. > > There's no BDFL delegation in place for this one though, so it's > really Guido's +1 that you need :) > > Cheers, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
On Wed, Jul 13, 2016 at 7:15 AM, Martin Teichmann wrote: > Hi list, > > another round for PEP 487, is there any chance it still makes it into > Python 3.6? Sure, feature freeze isn't until September (https://www.python.org/dev/peps/pep-0494/). > The PEP should be effectively done, I updated the examples in it, > given that I implemented the PEP I could actually test the examples, > so now they work. I am +1 on the idea of the PEP; below I am just asking for some clarifications and pointing out a typo or two. Please submit the next version to the github peps project as a PR! Re-review should be much quicker then. > The implementation is at http://bugs.python.org/issue27366, including > documentation and tests. Unfortunately nobody has reviewed the patch > yet. Sorry, I don't have time for that part, but I'm sure once the PEP is approved the review will follow. > The new version of the PEP is attached. > > Greetings > > Martin > > PEP: 487 > Title: Simpler customisation of class creation > Version: $Revision$ > Last-Modified: $Date$ > Author: Martin Teichmann , > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 27-Feb-2015 > Python-Version: 3.6 > Post-History: 27-Feb-2015, 5-Feb-2016, 24-Jun-2016, 2-Jul-2016, 13-Jul-2016 > Replaces: 422 > > > Abstract > > > Currently, customising class creation requires the use of a custom metaclass. > This custom metaclass then persists for the entire lifecycle of the class, > creating the potential for spurious metaclass conflicts. > > This PEP proposes to instead support a wide range of customisation > scenarios through a new ``__init_subclass__`` hook in the class body, > and a hook to initialize attributes. > > The new mechanism should be easier to understand and use than > implementing a custom metaclass, and thus should provide a gentler > introduction to the full power of Python's metaclass machinery. > > > Background > == > > Metaclasses are a powerful tool to customize class creation. They have, > however, the problem that there is no automatic way to combine metaclasses. > If one wants to use two metaclasses for a class, a new metaclass combining > those two needs to be created, typically manually. > > This need often occurs as a surprise to a user: inheriting from two base > classes coming from two different libraries suddenly raises the necessity > to manually create a combined metaclass, where typically one is not > interested in those details about the libraries at all. This becomes > even worse if one library starts to make use of a metaclass which it > has not done before. While the library itself continues to work perfectly, > suddenly every code combining those classes with classes from another library > fails. > > Proposal > > > While there are many possible ways to use a metaclass, the vast majority > of use cases falls into just three categories: some initialization code > running after class creation, the initalization of descriptors and initialization > keeping the order in which class attributes were defined. > > The first two categories can easily be achieved by having simple hooks > into the class creation: > > 1. An ``__init_subclass__`` hook that initializes >all subclasses of a given class. > 2. upon class creation, a ``__set_owner__`` hook is called on all the >attribute (descriptors) defined in the class, and > > The third category is the topic of another PEP 520. PEP, PEP 520. > As an example, the first use case looks as follows:: > >>>> class QuestBase: >...# this is implicitly a @classmethod maybe add "(see below for motivation)" ? >...def __init_subclass__(cls, swallow, **kwargs): >...cls.swallow = swallow >...super().__init_subclass__(**kwargs) > >>>> class Quest(QuestBase, swallow="african"): >...pass > >>>> Quest.swallow >'african' > > The base class ``object`` contains an empty ``__init_subclass__`` > method which serves as an endpoint for cooperative multiple inheritance. > Note that this method has no keyword arguments, meaning that all > methods which are more specialized have to process all keyword > arguments. > > This general proposal is not a new idea (it was first suggested for > inclusion in the language definition `more than 10 years ago`_, and a > similar mechanism has long been supported by `Zope's ExtensionClass`_), > but the situation has changed sufficiently in recent years that > the idea is worth reconsidering for inclusion. > > The second part of the proposal adds an ``__set_owner__`` > initializer for class attributes, especially if they are descriptors. > Descriptors are defined in the body of a > class, but they do not know anything about that class, they do not > even know the name they are accessed with. They do get to know their > owner once ``__get__`` is called, but still they do not know their > name. This is unfortunate, for example they cannot put their > associated value
Re: [Python-Dev] PEP487: Simpler customization of class creation
FWIW I copied the version you posted into the peps repo already, since it provides a significant update to the last version there. On Wed, Jul 13, 2016 at 2:02 PM, Guido van Rossum wrote: > I'm reviewing this now. > > Martin, can you please submit the new version of your PEP as a Pull > Request to the new peps repo on GitHub? https://github.com/python/peps > > --Guido > > On Wed, Jul 13, 2016 at 7:45 AM, Nick Coghlan wrote: >> On 14 July 2016 at 00:15, Martin Teichmann wrote: >>> Hi list, >>> >>> another round for PEP 487, is there any chance it still makes it into >>> Python 3.6? >>> >>> The PEP should be effectively done, I updated the examples in it, >>> given that I implemented the PEP I could actually test the examples, >>> so now they work. >>> >>> The implementation is at http://bugs.python.org/issue27366, including >>> documentation and tests. Unfortunately nobody has reviewed the patch >>> yet. >>> >>> The new version of the PEP is attached. >> >> +1 from me for this version - between them, this and PEP 520 address >> everything I hoped to achieve with PEP 422, and a bit more besides. >> >> There's no BDFL delegation in place for this one though, so it's >> really Guido's +1 that you need :) >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | [email protected] | Brisbane, Australia >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > -- > --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
