Re: [Python-Dev] PEP487: Simpler customization of class creation
On 14 July 2016 at 08:46, Guido van Rossum wrote:
> On Wed, Jul 13, 2016 at 7:15 AM, Martin Teichmann
> wrote:
>> Another small change should be noted here: in the current implementation of
>> CPython, ``type.__init__`` explicitly forbids the use of keyword arguments,
>> while ``type.__new__`` allows for its attributes to be shipped as keyword
>> arguments. This is weirdly incoherent, and thus the above code forbids that.
>> While it would be possible to retain the current behavior, it would be better
>> if this was fixed, as it is probably not used at all: the only use case would
>> be that at metaclass calls its ``super().__new__`` with *name*, *bases* and
>> *dict* (yes, *dict*, not *namespace* or *ns* as mostly used with modern
>> metaclasses) as keyword arguments. This should not be done.
>>
>> As a second change, the new ``type.__init__`` just ignores keyword
>> arguments. Currently, it insists that no keyword arguments are given. This
>> leads to a (wanted) error if one gives keyword arguments to a class
>> declaration
>> if the metaclass does not process them. Metaclass authors that do want to
>> accept keyword arguments must filter them out by overriding ``__init___``.
>>
>> In the new code, it is not ``__init__`` that complains about keyword
>> arguments,
>> but ``__init_subclass__``, whose default implementation takes no arguments.
>> In
>> a classical inheritance scheme using the method resolution order, each
>> ``__init_subclass__`` may take out it's keyword arguments until none are
>> left,
>> which is checked by the default implementation of ``__init_subclass__``.
>
> I called this out previously, and I am still a bit uncomfortable with
> the backwards incompatibility here. But I believe what you describe
> here is the compromise proposed by Nick, and if that's the case I have
> peace with it.
It would be worth spelling out the end result of the new behaviour in
the PEP to make sure it's what we want. Trying to reason about how
that code works is difficult, but looking at some class definition
scenarios and seeing how they behave with the old semantics and the
new semantics should be relatively straightforward (and they can
become test cases for the revised implementation).
The basic scenario to cover would be defining a metaclass which
*doesn't* accept any additional keyword arguments and seeing how it
fails when passed an unsupported parameter:
class MyMeta(type):
pass
class MyClass(metaclass=MyMeta, otherarg=1):
pass
MyMeta("MyClass", (), otherargs=1)
import types
types.new_class("MyClass", (), dict(metaclass=MyMeta, otherarg=1))
types.prepare_class("MyClass", (), dict(metaclass=MyMeta, otherarg=1))
Current behaviour:
>>> class MyMeta(type):
... pass
...
>>> class MyClass(metaclass=MyMeta, otherarg=1):
... pass
...
Traceback (most recent call last):
File "", line 1, in
TypeError: type() takes 1 or 3 arguments
>>> MyMeta("MyClass", (), otherargs=1)
Traceback (most recent call last):
File "", line 1, in
TypeError: Required argument 'dict' (pos 3) not found
>>> import types
>>> types.new_class("MyClass", (), dict(metaclass=MyMeta, otherarg=1))
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.5/types.py", line 57, in new_class
return meta(name, bases, ns, **kwds)
TypeError: type() takes 1 or 3 arguments
>>> types.prepare_class("MyClass", (), dict(metaclass=MyMeta, otherarg=1))
(, {}, {'otherarg': 1})
The error messages may change, but the cases which currently fail
should continue to fail with TypeError
Further scenarios would then cover the changes needed to the
definition of "MyMeta" to make the class creation invocations above
actually work (since the handling of __prepare__ already tolerates
unknown arguments).
First, just defining __new__ (which currently fails):
>>> class MyMeta(type):
... def __new__(cls, name, bases, namespace, otherarg):
... self = super().__new__(cls, name, bases, namespace)
... self.otherarg = otherarg
... return self
...
>>> class MyClass(metaclass=MyMeta, otherarg=1):
... pass
...
Traceback (most recent call last):
File "", line 1, in
TypeError: type.__init__() takes no keyword arguments
Making this work would be fine, and that's what I believe will happen
with the PEP's revised semantics.
Then, just defining __init__ (which also fails):
>>> class MyMeta(type):
... def __init__(self, name, bases, namespace, otherarg):
... super().__init__(name, bases, namespace)
... self.otherarg = otherarg
...
>>> class MyClass(metaclass=MyMeta, otherarg=1):
... pass
...
Traceback (most recent call last):
File "", line 1, in
TypeError: type() takes 1 or 3 arguments
The PEP shouldn't result in any changes in this case.
And finally defining both of them (which succeeds):
>>> class MyMeta(type):
... def __new__(cls, name, bases, namespace, otherarg):
... self = super().__new__(cls, name, base
Re: [Python-Dev] PEP487: Simpler customization of class creation
Hi Guido, Hi list, Thanks for the nice review! I applied followed up your ideas and put it into a github pull request: https://github.com/python/peps/pull/53 Soon we'll be working there, until then, some responses to your comments: > I wonder if this should be renamed to __set_name__ or something else > that clarifies we're passing it the name of the attribute? The method > name __set_owner__ made me assume this is about the owning object > (which is often a useful term in other discussions about objects), > whereas it is really about telling the descriptor the name of the > attribute for which it applies. The name for this has been discussed a bit already, __set_owner__ was Nick's idea, and indeed, the owner is also set. Technically, __set_owner_and_name__ would be correct, but actually I like your idea of __set_name__. > That (inheriting type from type, and object from object) is very > confusing. Why not just define new classes e.g. NewType and NewObject > here, since it's just pseudo code anyway? Actually, it's real code. If you drop those lines at the beginning of the tests for the implementation (as I have done here: https://github.com/tecki/cpython/blob/pep487b/Lib/test/test_subclassinit.py), the test runs on older Pythons. But I see that my idea to formulate things here in Python was a bad idea, I will put the explanation first and turn the code into pseudo-code. >> def __init__(self, name, bases, ns, **kwargs): >> super().__init__(name, bases, ns) > > What does this definition of __init__ add? It removes the keyword arguments. I describe that in prose a bit down. >> class object: >> @classmethod >> def __init_subclass__(cls): >> pass >> >> class object(object, metaclass=type): >> pass > > Eek! Too many things named object. Well, I had to do that to make the tests run... I'll take that out. >> In the new code, it is not ``__init__`` that complains about keyword >> arguments, >> but ``__init_subclass__``, whose default implementation takes no arguments. >> In >> a classical inheritance scheme using the method resolution order, each >> ``__init_subclass__`` may take out it's keyword arguments until none are >> left, >> which is checked by the default implementation of ``__init_subclass__``. > > I called this out previously, and I am still a bit uncomfortable with > the backwards incompatibility here. But I believe what you describe > here is the compromise proposed by Nick, and if that's the case I have > peace with it. No, this is not Nick's compromise, this is my original. Nick just sent another mail to this list where he goes a bit more into the details, I'll respond to that about this topic. Greetings Martin P.S.: I just realized that my changes to the PEP were accepted by someone else than Guido. I am a bit surprised about that, but I guess this is how it works? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
Hi Nick, hi list, > It would be worth spelling out the end result of the new behaviour in > the PEP to make sure it's what we want. Trying to reason about how > that code works is difficult, but looking at some class definition > scenarios and seeing how they behave with the old semantics and the > new semantics should be relatively straightforward (and they can > become test cases for the revised implementation). I agree with that. All the examples that Nick gives should work the same way as they used to. I'll turn them into tests. I hope it is fine if the error messages change slightly, as long as the error type stays the same? Let's look at an example that won't work anymore if my proposal goes throught: class MyType(type): def __new__(cls, name, bases, namespace): return super().__new__(cls, name=name, bases=bases, dict=namespace) I guess this kind of code is pretty rare. Note that I need to call the third parameter dict, as this is how it is named. Even if there is code out there like that, it would be pretty easy to change. Just in case someone wondered: def __new__(cls, **kwargs): return super().__new__(cls, **kwargs) doesn't work now and won't work afterwards, as the interpreter calls the metaclass with positional arguments. That said, it would be possible, at the cost of quite some lines of code, to make it fully backwards compatible. If the consensus is that this is needed, I'll change the PEP and code accordingly. My proposal also has the advantage that name, bases and dict may be used as class keyword arguments. At least for name I see a usecase: class MyMangledClass(BaseClass, name="Nice class name"): pass Greetings Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
I just reviewed the changes you made, I like __set_name__(). I'll just wait for your next update, incorporating Nick's suggestions. Regarding who merges PRs to the PEPs repo, since you are the author the people who merge don't pass any judgment on the changes (unless it doesn't build cleanly or maybe if they see a typo). If you intend a PR as a base for discussion you can add a comment saying e.g. "Don't merge yet". If you call out @gvanrossum, GitHub will make sure I get a message about it. I think the substantial discussion about the PEP should remain here in python-dev; comments about typos, grammar and other minor editorial issues can go on GitHub. Hope this part of the process makes sense! On Thu, Jul 14, 2016 at 6:50 AM, Martin Teichmann wrote: > Hi Guido, Hi list, > > Thanks for the nice review! I applied followed up your ideas and put > it into a github pull request: https://github.com/python/peps/pull/53 > > Soon we'll be working there, until then, some responses to your comments: > >> I wonder if this should be renamed to __set_name__ or something else >> that clarifies we're passing it the name of the attribute? The method >> name __set_owner__ made me assume this is about the owning object >> (which is often a useful term in other discussions about objects), >> whereas it is really about telling the descriptor the name of the >> attribute for which it applies. > > The name for this has been discussed a bit already, __set_owner__ was > Nick's idea, and indeed, the owner is also set. Technically, > __set_owner_and_name__ would be correct, but actually I like your idea > of __set_name__. > >> That (inheriting type from type, and object from object) is very >> confusing. Why not just define new classes e.g. NewType and NewObject >> here, since it's just pseudo code anyway? > > Actually, it's real code. If you drop those lines at the beginning of > the tests for the implementation (as I have done here: > https://github.com/tecki/cpython/blob/pep487b/Lib/test/test_subclassinit.py), > the test runs on older Pythons. > > But I see that my idea to formulate things here in Python was a bad > idea, I will put the explanation first and turn the code into > pseudo-code. > >>> def __init__(self, name, bases, ns, **kwargs): >>> super().__init__(name, bases, ns) >> >> What does this definition of __init__ add? > > It removes the keyword arguments. I describe that in prose a bit down. > >>> class object: >>> @classmethod >>> def __init_subclass__(cls): >>> pass >>> >>> class object(object, metaclass=type): >>> pass >> >> Eek! Too many things named object. > > Well, I had to do that to make the tests run... I'll take that out. > >>> In the new code, it is not ``__init__`` that complains about keyword >>> arguments, >>> but ``__init_subclass__``, whose default implementation takes no arguments. >>> In >>> a classical inheritance scheme using the method resolution order, each >>> ``__init_subclass__`` may take out it's keyword arguments until none are >>> left, >>> which is checked by the default implementation of ``__init_subclass__``. >> >> I called this out previously, and I am still a bit uncomfortable with >> the backwards incompatibility here. But I believe what you describe >> here is the compromise proposed by Nick, and if that's the case I have >> peace with it. > > No, this is not Nick's compromise, this is my original. Nick just sent > another mail to this list where he goes a bit more into the details, > I'll respond to that about this topic. > > Greetings > > Martin > > P.S.: I just realized that my changes to the PEP were accepted by > someone else than Guido. I am a bit surprised about that, but I guess > this is how it works? > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __qualname__ exposed as a local variable: standard?
I think this and similar issues suffer from a lack of people who have both the time and the understanding to review patches and answer questions of this nature. To the OP: I would recommend not depending on the presence of __qualname__ in the locals, as another compiler/interpreter may come up with a different approach. However, if you have a use case, please present it here and maybe we could consider making this a documented feature. To Martin: it would be easier for people (even myself, who implemented this super() hack eons ago) to review your patch if you were able to explain the current and proposed behavior more precisely. You are doing your best but realistically almost nobody has enough context to understand what you wrote in the tracker (certainly to me it's not enough to remind me of how the super() machinery currently works -- but reading the patch doesn't enlighten me much either as it's mostly about changes to the low-level C code). On Wed, Jul 13, 2016 at 7:00 AM, Martin Teichmann wrote: > Hi list, > >> I noticed __qualname__ is exposed by locals() while defining a class. This >> is handy but I'm not sure about its status: is it standard or just an >> artifact of the current implementation? (btw, the pycodestyle linter -former >> pep8- rejects its usage). I was unable to find any reference to this >> behavior in PEP 3155 nor in the language reference. > > I would like to underline the importance of this question, and give > some background, as it happens to resonate with my work on PEP 487. > > The __qualname__ of a class is originally determined by the compiler. > One might now think that it would be easiest to simply set the > __qualname__ of a class once the class is created, but this is not as > easy as it sounds. The class is created by its metaclass, so possibly > by user code, which might create whatever it wants, including > something which is not even a class. So the decision had been taken to > sneak the __qualname__ through user code, and pass it to the > metaclasses __new__ method as part of the namespace, where it is > deleted from the namespace again. This has weird side effects, as the > namespace may be user code as well, leading to the funniest possible > abuses, too obscene to publish on a public mailing list. > > A very different approach has been taken for super(). It has similar > problems: the zero argument version of super looks in the surrounding > scope for __class__ for the containing class. This does not exist yet > at the time of creation of the methods, so a PyCell is put into the > function's scope, which will later be filled. It is actually filled > with whatever the metaclasses __new__ returns, which may, as already > be said, anything (some sanity checks are done to avoid crashing the > interpreter). > > I personally prefer the first way of doing things like for > __qualname__, even at the cost of adding things to the classes > namespace. It could be moved after the end of the class definition, > such that it doesn't show up while the class body is executed. We > might also rename it to __@qualname__, this way it cannot be accessed > by users in the class body, unless they look into locals(). > > This has the large advange that super() would work immediately after > the class has been defined, i.e. already in the __new__ of the > metaclass after it has called type.__new__. > > All of this changes the behavior of the interpreter, but we are > talking about undocumented behavior. > > The changes necessary to make super() work earlier are store in > http://bugs.python.org/issue23722 > > Greetings > > Martin > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/14/2016 11:47 AM, Guido van Rossum wrote: > If you intend a PR as a base for discussion you can add a comment > saying e.g. "Don't merge yet". If you call out @gvanrossum, GitHub > will make sure I get a message about it. FWIW, I often use a Github label, "don't merge" (colored red for urgency), to indicate that PRs are still in discussion stage: removing it is a lightweight way to signify that blocking issues have been resolved (in the opinion of the owner/matintainer, anyway). Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJXh9P8AAoJEPKpaDSJE9HYsC0QAMPf/J+n35fg7sMjy07/fE8v xXXc+JbqnphnZolX1Xjla8sUD6tSpq0dp234VYeGwm4z19p3U5SYpYX4zzNuYCwE V2AVfdNpY0xwTkoFCbxXTwikZVIGt6o8IQLvqcjlQpCj3wl5A0ggcoaYDnXeKrZd wE4MF4t9YFgdABZ2i2RVbZNoSRUcMa1kKq9BKpnLnq65dPv2yAYQDDbWIatXLLbi 7kxAfK4CjSWR8BKNzo71uJDeVJVyk6N2nWLuGNOEff8BVZe83cG/2SRjRGALSb0h kV6FdPhwIhoZ+KrVvkLcbJYpUykBAPK68VSnomXNU14jpY9a3zqEIrirB4YLM3tS 9Ov2GYH+AhDPQ840B197mmkGN4nu/d52jCHPfecgccz2gooy+qoK3FRrMlshTTaD dbnTlNm/mkEBad8dz7l/u7cGvVG+k5AiFCGkOMikg4So0xXw7C9ulCQhoARWa0DS J0gTqEGHzGqYAwMXvWxobvlm3HxcxutWuYYx7vD0DRKrPRdpz/ELE7XpOh5bPjjU sEpt7gaAn/q962QorCDRopvqgd7MeRkrAdPKJzhCIeSUp9+Y/oqolZ/my4uEXSju W8WHWx41ioDvoUEHFW3pYljSN075STP21SCuxJh+GBDOVS2HsMXEb09wxM81GOAt V/mBLuZeptsVMiVSQk6J =/KS/ -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
On Thu, 14 Jul 2016 at 11:05 Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07/14/2016 11:47 AM, Guido van Rossum wrote: > > > If you intend a PR as a base for discussion you can add a comment > > saying e.g. "Don't merge yet". If you call out @gvanrossum, GitHub > > will make sure I get a message about it. > > FWIW, I often use a Github label, "don't merge" (colored red for > urgency), to indicate that PRs are still in discussion stage: removing > it is a lightweight way to signify that blocking issues have been > resolved (in the opinion of the owner/matintainer, anyway). > Just start the title with `[WIP]` and it will be obvious that it's a work-in-progress (it's a GitHub idiom). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP487: Simpler customization of class creation
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/14/2016 02:10 PM, Brett Cannon wrote: > On Thu, 14 Jul 2016 at 11:05 Tres Seaver > wrote: > >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> >> On 07/14/2016 11:47 AM, Guido van Rossum wrote: >> >>> If you intend a PR as a base for discussion you can add a comment >>> saying e.g. "Don't merge yet". If you call out @gvanrossum, >>> GitHub will make sure I get a message about it. >> >> FWIW, I often use a Github label, "don't merge" (colored red for >> urgency), to indicate that PRs are still in discussion stage: >> removing it is a lightweight way to signify that blocking issues >> have been resolved (in the opinion of the owner/matintainer, >> anyway). >> > > Just start the title with `[WIP]` and it will be obvious that it's a > work-in-progress (it's a GitHub idiom). De gustibus, I guess: unlike the title, labels stay visible no matter how one scrolls the PR / issue, and they are more easily searchable. Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJXh90aAAoJEPKpaDSJE9HYeFMP/3r+b6MP4VI+SLnPT6PeZdHU aVn9/bz4hMb4DtIB1adp6CBEdtxijg0Y2H6BgcmnoFcqhVO0yXquOtJmVfqQr44T zO8DY+v+eVBcw8KX5MduQt3jLq8fBXviFq0yu55bWYboQRKbUKrfzFwZFlZJ9gH7 AAdieX/26NK4RkFxePYn5dJeJ1EIX7RoRuIB8X5NPve6FA08eRUHvSicQN4Vpvey Xs+eiLcz+3pOHCu4hiERInu19lztoL5GmdC+cL3mq2A9qpKy9fEAWVRhU84VaDa1 86/jKXgoXfZt2wH7Wj/MC6Z8gXMutIyjcrjVyZEbPQe4zt5o5Vdv/M9nxk1iOnV3 sSqY72HQiiaWvwjWasv0F78LT0nKqt9+bq+aBHrF5PHd0epxInI7KQEScuB+BcaS aNNVZtSRRQhCEnO8MB6cedBv90sg2FVv8ITBNHac/Zn2ThljMJ8s90gHZZbC3T6/ uP0uvwS8aYzKJoTH5Mmxvt4m4vQCg+tintOwF8/nwN4y4kQFXZcCZqeb4l55XRAE INal/Khx0eHqd07D7BRZ/a1lKTDuyEuTifJNjZjr9fC704xplMTygJc/kuaTvMfN 4e30iKbMO4oJ3Oyrysr/2E81YlqBe9ZMGdkdBwvyYmGnIKXbmlsHHUQn1asRwF64 l5HJUWDAWxccJ8d83q0g =NdlU -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
