Re: [Python-Dev] Replacing self.__dict__ in __init__
That's reassuring, thanks. On Sat, Mar 24, 2018 at 5:20 PM Raymond Hettinger < [email protected]> wrote: > This should work. I've seen it done in other production tools without any > ill effect. > > The dict can be replaced during __init__() and still get benefits of > key-sharing. That benefit is lost only when the instance dict keys are > modified downstream from __init__(). So, from a dict size point of view, > your optimization is fine. > > Still, you should look at whether this would affect static type checkers, > lint tools, and other tooling. > > > Raymond ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Use more Argument Clinic Annotations?
Hi Python Devs I recently started testing Jedi with Python 3.7. Some tests broke. I realized that one of the things that changed in 3.7 was the use of argument clinic in methods like str.replace. The issue is that the text signature doesn't contain a return annotation. >>> str.replace.__text_signature__ '($self, old, new, count=-1, /) In Python < 3.7 there was a `S.replace(old, new[, count]) -> str` at the top of the __doc__. T If the __text_signature__ was `'($self, old, new, count=-1, /) -> str` a lot of tools would be able to have the information again. Is this intentional or was this just forgotten? I'd like to note that this information is insanely helpful (at least for Jedi) to pick up type information. I really hope this information can make it back into 3.7, since it was there in earlier versions. If you lack don't have time I might have some. Just give me some instructions. ~ Dave PS: Don't get me wrong, I love argument clinic/inspect.signature and am generally in favor of using it everywhere. It helps tools like jedi, pycharm and others get accurate information about a builtin function. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacing self.__dict__ in __init__
On Sun, Mar 25, 2018 at 5:23 AM Nick Coghlan wrote: > That depends on what you mean by "safe" :) > > It won't crash, but it will lose any existing entries that a metaclass, > subclass, or __new__ method implementation might have added to the instance > dictionary before calling the __init__ method. That can be OK in a tightly > controlled application specific class hierarchy, but it would be > questionable in a general purpose utility library that may be combined with > arbitrary other types. > > As Kirill suggests, `self.__dict__.update(new_attrs)` is likely to be > faster than repeated assignment statements, without the potentially odd > interactions with other instance initialisation code. > > It should also be explicitly safe to do in the case of "type(self) is > __class__ and not self.__dict__", which would let you speed up the common > case of direct instantiation, while falling back to the update based > approach when combined with other classes at runtime. > > Hm, food for thought, thank you. The entire point of the exercise is to shave nanoseconds off of __init__. Using Victor Stinner's excellent pyperf tool and CPython 3.6.3 on Linux, I see the dict replacement approach always beating the series of assignments approach, and the update approach always losing to the series of assignments. For example, for a simple class with 9 attributes: Series of assignments: Mean +- std dev: 1.31 us +- 0.06 us Dict replacement: Mean +- std dev: 1.04 us +- 0.04 us Dict update: Mean +- std dev: 1.67 us +- 0.06 us Nick's guard: 1.34 us +- 0.03 us Based on these numbers, I don't think the update approach and the guard approach are worth doing. The dict replacement approach is 30% faster though, so it's hard to ignore. The attrs generated __init__ was always a little special, for example it never calls super().__init__. Now we just need to figure out how common are the special cases you called out, and whether to make this vroom-vroom init opt-in or opt-out. Kind regards, Tin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use more Argument Clinic Annotations?
25.03.18 15:36, Dave Halter пише: I recently started testing Jedi with Python 3.7. Some tests broke. I realized that one of the things that changed in 3.7 was the use of argument clinic in methods like str.replace. The issue is that the text signature doesn't contain a return annotation. str.replace.__text_signature__ '($self, old, new, count=-1, /) In Python < 3.7 there was a `S.replace(old, new[, count]) -> str` at the top of the __doc__. T If the __text_signature__ was `'($self, old, new, count=-1, /) -> str` a lot of tools would be able to have the information again. Is this intentional or was this just forgotten? I'd like to note that this information is insanely helpful (at least for Jedi) to pick up type information. I really hope this information can make it back into 3.7, since it was there in earlier versions. Argument Clinic convertors don't have any relations with annotations. Annotations are not supported by Argument Clinic. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacing self.__dict__ in __init__
25.03.18 18:38, Tin Tvrtković пише: For example, for a simple class with 9 attributes: What are results for classes with 2 or 100 attributes? What are results in Python 3.5? I think you are playing on thin ice. Your results depend on implementation details of the bytecode (in particularly on adding BUILD_CONST_KEY_MAP in 3.6). Other implementations use different bytecode or don't use bytecode at all. CPython can introduce new opcodes in future versions which will change your result drastically. And the straightforward way could become the most fast. I suggest you to not worry and just wait for more general optimizations in CPython interpreter. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use more Argument Clinic Annotations?
2018-03-25 18:38 GMT+02:00 Serhiy Storchaka : > 25.03.18 15:36, Dave Halter пише: >> >> I recently started testing Jedi with Python 3.7. Some tests broke. I >> realized that one of the things that changed in 3.7 was the use of >> argument clinic in methods like str.replace. >> >> The issue is that the text signature doesn't contain a return annotation. >> > str.replace.__text_signature__ >> >> '($self, old, new, count=-1, /) >> >> >> In Python < 3.7 there was a `S.replace(old, new[, count]) -> str` at >> the top of the __doc__. T >> >> If the __text_signature__ was `'($self, old, new, count=-1, /) -> str` >> a lot of tools would be able to have the information again. >> >> Is this intentional or was this just forgotten? I'd like to note that >> this information is insanely helpful (at least for Jedi) to pick up >> type information. I really hope this information can make it back into >> 3.7, since it was there in earlier versions. > > > Argument Clinic convertors don't have any relations with annotations. > Annotations are not supported by Argument Clinic. Is there a way though in which the __text_signature__ could contain the information `-> str` or do we need to engineer that first? IMO it's just a small thing in which Python 3.7 got worse than 3.6 and I hope we can still fix that. Everything else looks great. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use more Argument Clinic Annotations?
25.03.18 19:47, Dave Halter пише: Is there a way though in which the __text_signature__ could contain the information `-> str` or do we need to engineer that first? There is no such way currently. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacing self.__dict__ in __init__
On Sun, Mar 25, 2018 at 9:51 AM, Serhiy Storchaka wrote: > 25.03.18 18:38, Tin Tvrtković пише: >> >> For example, for a simple class with 9 attributes: > What are results for classes with 2 or 100 attributes? What are results in > Python 3.5? > > I think you are playing on thin ice. Your results depend on implementation > details of the bytecode > > I suggest you to not worry and just wait for more general optimizations in > CPython interpreter. Indeed, sometimes strange code that was once faster, is made slower than the favored way by a future version of your favorite python interpreter. I hope there are more important things to worry about? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use more Argument Clinic Annotations?
2018-03-25 5:36 GMT-07:00 Dave Halter : > Hi Python Devs > > I recently started testing Jedi with Python 3.7. Some tests broke. I > realized that one of the things that changed in 3.7 was the use of > argument clinic in methods like str.replace. > > The issue is that the text signature doesn't contain a return annotation. > > >>> str.replace.__text_signature__ > '($self, old, new, count=-1, /) > > > In Python < 3.7 there was a `S.replace(old, new[, count]) -> str` at > the top of the __doc__. T > > If the __text_signature__ was `'($self, old, new, count=-1, /) -> str` > a lot of tools would be able to have the information again. > > Is this intentional or was this just forgotten? I'd like to note that > this information is insanely helpful (at least for Jedi) to pick up > type information. I really hope this information can make it back into > 3.7, since it was there in earlier versions. > > If you lack don't have time I might have some. Just give me some > instructions. > > Perhaps you should use https://github.com/python/typeshed/ to get type information? > ~ Dave > > > PS: Don't get me wrong, I love argument clinic/inspect.signature and > am generally in favor of using it everywhere. It helps tools like > jedi, pycharm and others get accurate information about a builtin > function. > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > jelle.zijlstra%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Use more Argument Clinic Annotations?
On 03/25/2018 09:58 AM, Serhiy Storchaka wrote: 25.03.18 19:47, Dave Halter пише: Is there a way though in which the __text_signature__ could contain the information `-> str` or do we need to engineer that first? There is no such way currently. Are you sure? I'm pretty sure Argument Clinic signatures support "return converters", which are emitted in the text signature as a return annotation. See the section "Using a return converter" in Doc/howto/clinic.rst. What appears to be lacking is a "return converter" that handles strings. I don't know how easy or hard it would be to write one--I haven't thought about Argument Clinic in years. However, the DecodeFSDefault() returns a string type, with some extra implied constraints on the value I think. So my guess is, it wouldn't be too hard to add a simple "str" return converter. Cheers, //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacing self.__dict__ in __init__
>
> The dict can be replaced during __init__() and still get benefits of
> key-sharing. That benefit is lost only when the instance dict keys are
> modified downstream from __init__(). So, from a dict size point of view,
> your optimization is fine.
>
I think replacing __dict__ lose key-sharing.
Python 3.6.4 (default, Mar 9 2018, 23:15:03)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
... def __init__(self, a, b, c):
... self.a, self.b, self.c = a, b, c
...
>>> class D:
... def __init__(self, a, b, c):
... self.__dict__ = {'a':a, 'b':b, 'c':c}
...
>>> import sys
>>> sys.getsizeof(C(1,2,3).__dict__)
112
>>> sys.getsizeof(D(1,2,3).__dict__)
240
--
INADA Naoki
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Replacing self.__dict__ in __init__
On Mar 25, 2018, at 8:08 AM, Tin Tvrtković wrote: > > That's reassuring, thanks. I misspoke. The object size is the same but the underlying dictionary loses key-sharing and doubles in size. Raymond ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
