[Python-Dev] Intricacies of calling __eq__
Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] where d is a dict would result in only one dict lookup (the second one being constant folded away). The question is whether it's ok to do it, despite the fact that it changes the semantics on how many times __eq__ is called on x. Cheers, fijal ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] default hg.python.org/cpython is now 3.5
Hi, 2014-03-18 7:19 GMT+01:00 Georg Brandl : > Am 17.03.2014 22:36, schrieb Victor Stinner: >> I modified the Misc/NEWS file: (...) Is that correct? > > The changes not merged in 3.4.0 will all be in 3.5.0; please reinstate the > NEWS entries under the 3.5 heading. Oh ok, I didn't notice that 3.4.1rc1 items were all missing from 3.5. Does it look better with the following changeset? http://hg.python.org/cpython/rev/b7f2b1bd49f3 I compared Misc/NEWS with Misc/NEWS of 3.4 branch: the only change is the 2 new changes specific to 3.5. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Add a stub "whatsnew in 3.5" document.
2014-03-18 7:22 GMT+01:00 Georg Brandl : > Am 18.03.2014 01:27, schrieb victor.stinner: >> http://hg.python.org/cpython/rev/daa6bf71170f >> changeset: 89835:daa6bf71170f >> user:Victor Stinner >> date:Tue Mar 18 00:53:32 2014 +0100 >> summary: >> Add a stub "whatsnew in 3.5" document. >> >> files: >> Doc/whatsnew/3.5.rst | 186 +++ >> 1 files changed, 186 insertions(+), 0 deletions(-) > > Nice, but please also add it to the toctree in Doc/whatsnew/index.rst. Ok, done. Is there a document explaining the steps after a release to prepare the next release? Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg branching + log question
I have also found hg difficult to get to grips with from cold (but I like it). The hg command and its help are good, as Antoine says, but if I'm doing something complex, the visualisation of the change sets that TortoiseHG provides is invaluable (and of other invisible structures, such as the MQ patch stack). The context menus are also a clue to what you might want to do next when you can't guess what word comes after hg help ... . I found it helpful to practice extensively on something that doesn't matter. The gap for me is still examples of what I want "done well". Clearly the Python repos represent complex work, but even accepting it is all done well, are without much commentary. This is very good: http://hgbook.red-bean.com/read/ , but there are hints it has not kept up. This also: http://legacy.python.org/dev/peps/pep-0385/ Jeff Allen On 17/03/2014 23:53, Sean Felipe Wolfe wrote: Ah well, ok. That seems pretty counterintuitive to me though. I suppose Hg has its quirks just like ... that other DCVS system ... :P On Mon, Mar 17, 2014 at 1:07 PM, Antoine Pitrou wrote: On Mon, 17 Mar 2014 13:02:23 -0700 Sean Felipe Wolfe wrote: I'm getting my feet wet with the cpython sources and Mercurial. I'm a bit confused -- when I checkout a branch, eg. 3.3, and I do an 'hg log', why do I see log messages for other branches? This is a classic hg question, you would get the answer by asking Mercurial for help: hg log --help :) Basically, to restrict the log to a given branch, just use the -b option: hg log -b 3.3. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ether.joe%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
On 18 Mar 2014 11:56, "Victor Stinner" wrote:
>
> Hi,
>
> I modified Python 3.5 to use the "surrogateescape" error handler (PEP
> 383) for stdin and stdout when the LC_CTYPE locale is POSIX ("C"
> locale):
> http://bugs.python.org/issue19977
Yay, thanks Victor. I'll let the Fedora folks know this has been merged, as
we may seriously consider applying this as a vendor patch to our build of
Python 3.4 (while I agree this isn't a bug fix, the current behaviour also
poses a problem for Fedora as more core utilities start migrating to Python
3).
Cheers,
Nick.
>
> New behaviour:
> ---
> $ mkdir z
> $ touch z/abcé
> $ LC_CTYPE=C ./python -c 'import os; print(os.listdir("z")[0])'
> abcé
> ---
>
> Old behaviour, before the change (test with Python 3.3):
> ---
> $ LC_CTYPE=C python3 -c 'import os; print(os.listdir("z")[0])'
> Traceback (most recent call last):
> File "", line 1, in
> UnicodeEncodeError: 'ascii' codec can't encode characters in position
> 3-4: ordinal not in range(128)
> ---
>
> The POSIX locale is common because it is used by default when no other
> locale is set. It's common that programs started by a crontab on UNIX
> and daemons are using this locale.
>
> Victor
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Add a stub "whatsnew in 3.5" document.
Am 18.03.2014 09:02, schrieb Victor Stinner: > 2014-03-18 7:22 GMT+01:00 Georg Brandl : >> Am 18.03.2014 01:27, schrieb victor.stinner: >>> http://hg.python.org/cpython/rev/daa6bf71170f >>> changeset: 89835:daa6bf71170f >>> user:Victor Stinner >>> date:Tue Mar 18 00:53:32 2014 +0100 >>> summary: >>> Add a stub "whatsnew in 3.5" document. >>> >>> files: >>> Doc/whatsnew/3.5.rst | 186 +++ >>> 1 files changed, 186 insertions(+), 0 deletions(-) >> >> Nice, but please also add it to the toctree in Doc/whatsnew/index.rst. > > Ok, done. Is there a document explaining the steps after a release to > prepare the next release? Yes, in PEP 101, but it will never be 100% complete. The step of adding a new whatsnew document could be added, for example. Georg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 3/18/2014 3:52 AM, Maciej Fijalkowski wrote: Hi I have a question about calling __eq__ in some cases. We're thinking about doing an optimization where say: if x in d: return d[x] if d.__contains__(x): return d.__getitem__(x) I do not see any requirement to call x.__eq__ any particular number of times. The implementation of d might always call somekey.__eq__(x). The concept of sets (and dicts) requires coherent equality comparisons. where d is a dict would result in only one dict lookup (the second one being constant folded away). The question is whether it's ok to do it, despite the fact that it changes the semantics on how many times __eq__ is called on x. A __eq__ that has side-effects violates the intended and expected semanitics of __eq__. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
2014-03-18 9:08 GMT+01:00 Nick Coghlan :
> On 18 Mar 2014 11:56, "Victor Stinner" wrote:
>>
>> Hi,
>>
>> I modified Python 3.5 to use the "surrogateescape" error handler (PEP
>> 383) for stdin and stdout when the LC_CTYPE locale is POSIX ("C"
>> locale):
>> http://bugs.python.org/issue19977
>
> Yay, thanks Victor. I'll let the Fedora folks know this has been merged, as
> we may seriously consider applying this as a vendor patch to our build of
> Python 3.4 (while I agree this isn't a bug fix, the current behaviour also
> poses a problem for Fedora as more core utilities start migrating to Python
> 3).
Please don't cherry-pick this change in Fedora if it is not done in
Python 3.4. It changes the behaviour of Python and I would prefer to
have the same behaviour on the same Python version on all platforms.
I'm not against backporting the change in Python 3.4.1. It can be seen
as a bugfix. I don't think that anyone wants a Unicode error when
reading or printing non-ASCII data from stdin/to stdout. But I would
like the opinion of other developers before doing that.
Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 18 March 2014 17:52, Maciej Fijalkowski wrote: > Hi > > I have a question about calling __eq__ in some cases. > > We're thinking about doing an optimization where say: > > if x in d: >return d[x] > > where d is a dict would result in only one dict lookup (the second one > being constant folded away). The question is whether it's ok to do it, > despite the fact that it changes the semantics on how many times > __eq__ is called on x. I'll assume the following hold: - we're only talking about true builtin dicts (the similarity between __contains__ and __getitem__ can't be assumed otherwise) - guards will trigger if d is mutated (e.g. by another thread) between the containment check and the item retrieval Semantically, what you propose is roughly equivalent to reinterpreting the look-before-you-leap version to the exception handling based fallback: try: return d[x] except KeyError: pass For a builtin dict and any *reasonable* x, those two operations will behave the same way. Differences arise only if x.__hash__ or x.__eq__ is defined in a way that most people would consider unreasonable. For an optimisation that actually changes the language semantics like that, though, I would expect it to be buying a significant payoff in speed, especially given that most cases where the key lookup is known to be a bottleneck can already be optimised by hand. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
On 18 March 2014 19:13, Victor Stinner wrote:
> 2014-03-18 9:08 GMT+01:00 Nick Coghlan :
>> On 18 Mar 2014 11:56, "Victor Stinner" wrote:
>>>
>>> Hi,
>>>
>>> I modified Python 3.5 to use the "surrogateescape" error handler (PEP
>>> 383) for stdin and stdout when the LC_CTYPE locale is POSIX ("C"
>>> locale):
>>> http://bugs.python.org/issue19977
>>
>> Yay, thanks Victor. I'll let the Fedora folks know this has been merged, as
>> we may seriously consider applying this as a vendor patch to our build of
>> Python 3.4 (while I agree this isn't a bug fix, the current behaviour also
>> poses a problem for Fedora as more core utilities start migrating to Python
>> 3).
>
> Please don't cherry-pick this change in Fedora if it is not done in
> Python 3.4. It changes the behaviour of Python and I would prefer to
> have the same behaviour on the same Python version on all platforms.
>
> I'm not against backporting the change in Python 3.4.1. It can be seen
> as a bugfix. I don't think that anyone wants a Unicode error when
> reading or printing non-ASCII data from stdin/to stdout. But I would
> like the opinion of other developers before doing that.
Well, the concern has always been the risk of silently generating bad
data if there is a mismatch between the OS encoding and the stream
encodings. That's why it took so long to make this change at all - we
had to figure out that the underlying problem was really the ease with
which even a properly configured Linux systems could end up running
Python 3 code in the POSIX locale, and thus end up with improperly
configured standard streams. Enabling "surrogateescape" by default
only when the standard stream encoding is "ascii" helps to mitigate
that risk, while still dealing with the main problem. I meant to try
to get this into 3.4 (since a couple of the Fedora folks convinced me
it was a problem), but there are only so many hours in the day, and it
took me quite a while to fully grasp the actual problem.
If folks are open to backporting this change to 3.4.1, then yes, I'd
definitely prefer an upstream solution. Otherwise, it will be up to
the Fedora Python maintainers to decide what they want to do.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
2014-03-18 10:48 GMT+01:00 Nick Coghlan : > Well, the concern has always been the risk of silently generating bad > data if there is a mismatch between the OS encoding and the stream > encodings. Data can be loaded from OS functions, from files and from stdin. These 3 sources may use various different and incompatible encodings. surrogateescape is used by OS functions, and now also by stdin when the POSIX locale is used. When the POSIX locale is used, OS functions and stdin can use different encodings if the PYTHONIOENCODING environment variable is used. Since we are consentent adults, I guess that you understand what you are doing when you set PYTHONIOENCODING. On Windows, the encoding of standard streams is the OEM code page, or the ANSI code page if a stream is redirected, it's unrelated to the LC_CTYPE locale. So surrogateecape can be used when if the encoding of standard streams is not ASCII. We may handle Windows differently to use strict even if the LC_CTYPE locale is "C". Note: On FreeBSD, Solaris and OpenIndiana, nl_langinfo(CODESET) announces an alias of the ASCII encoding when the LC_CTYPE locale is POSIX, whereas mbstowcs() and wcstombs() functions use the ISO-8859-1 encoding. Python 3 now uses the ASCII encoding for its "filesystem" (OS) encoding. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
2014-03-18 11:02 GMT+01:00 Atsuo Ishimoto : > FYI: Guido was opposed to change error handler of stdin and stdout years ago. > > http://bugs.python.org/issue2630#msg65493 This issue proposes to use "backslashreplace" error handler for stdout. This error handler is very different to "surrogateescape" which is related to PEP 383 and used by all OS functions. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 11:35 AM, Nick Coghlan wrote: > On 18 March 2014 17:52, Maciej Fijalkowski wrote: >> Hi >> >> I have a question about calling __eq__ in some cases. >> >> We're thinking about doing an optimization where say: >> >> if x in d: >>return d[x] >> >> where d is a dict would result in only one dict lookup (the second one >> being constant folded away). The question is whether it's ok to do it, >> despite the fact that it changes the semantics on how many times >> __eq__ is called on x. > > I'll assume the following hold: > > - we're only talking about true builtin dicts (the similarity between > __contains__ and __getitem__ can't be assumed otherwise) yes > - guards will trigger if d is mutated (e.g. by another thread) between > the containment check and the item retrieval yes > > Semantically, what you propose is roughly equivalent to reinterpreting > the look-before-you-leap version to the exception handling based > fallback: > > try: > return d[x] > except KeyError: > pass > > For a builtin dict and any *reasonable* x, those two operations will > behave the same way. Differences arise only if x.__hash__ or x.__eq__ > is defined in a way that most people would consider unreasonable. > > For an optimisation that actually changes the language semantics like > that, though, I would expect it to be buying a significant payoff in > speed, especially given that most cases where the key lookup is known > to be a bottleneck can already be optimised by hand. > > Cheers, > Nick. the payoff is significant. Note that __eq__ might not be called at all (since dicts check identity first). It turns out not all people write reasonable code and we can't expect them to micro-optimize by hand. It also covers cases that are hard to optimize, like: if d[x] > 3: d[x] += 1 etc. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 05:05:56AM -0400, Terry Reedy wrote:
> On 3/18/2014 3:52 AM, Maciej Fijalkowski wrote:
> >Hi
> >
> >I have a question about calling __eq__ in some cases.
> >
> >We're thinking about doing an optimization where say:
> >
> >if x in d:
> >return d[x]
>
> if d.__contains__(x): return d.__getitem__(x)
[Aside: to be pedantic, Python only calls dunder methods on the class,
not the instance, in response to operators and other special calls. That
is, type(d).__contains__ rather than d.__contains__, etc. And to be even
more pedantic, that's only true for new-style classes.]
> I do not see any requirement to call x.__eq__ any particular number of
> times. The implementation of d might always call somekey.__eq__(x). The
> concept of sets (and dicts) requires coherent equality comparisons.
To what extent does Python the language specify that user-defined
classes must be coherent? How much latitude to shoot oneself in the foot
should the language allow?
What counts as coherent can depend on the types involved. For instance,
I consider IEEE-754 Not-A-Numbers to be coherent, albeit weird. Python
goes only so far to accomodate NANs: while it allows a NAN to test
unequal even to itself (`NAN == NAN` returns False), containers are
allowed to assume that instances are equal to themselves (`NAN in {NAN}`
returns True). This was discussed in great detail a few years ago, and
if I recall correctly, the conclusion was that containers can assume
that their elements are reflexive (they equal themselves), but equality
== cannot make the same assumption and bypass calling __eq__.
> >where d is a dict would result in only one dict lookup (the second one
> >being constant folded away). The question is whether it's ok to do it,
> >despite the fact that it changes the semantics on how many times
> >__eq__ is called on x.
>
> A __eq__ that has side-effects violates the intended and expected
> semanitics of __eq__.
Nevertheless, an __eq__ with side-effects is legal Python and may in
fact be useful.
It's a tricky one... I don't know how I feel about short-cutting normal
Python semantics for speed. On the one hand, faster is good. But on the
other hand, it makes it harder to reason about code when things go
wrong. "Why is my __eq__ method not being called?"
--
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 1:18 PM, Steven D'Aprano wrote:
> On Tue, Mar 18, 2014 at 05:05:56AM -0400, Terry Reedy wrote:
>> On 3/18/2014 3:52 AM, Maciej Fijalkowski wrote:
>> >Hi
>> >
>> >I have a question about calling __eq__ in some cases.
>> >
>> >We're thinking about doing an optimization where say:
>> >
>> >if x in d:
>> >return d[x]
>>
>> if d.__contains__(x): return d.__getitem__(x)
>
> [Aside: to be pedantic, Python only calls dunder methods on the class,
> not the instance, in response to operators and other special calls. That
> is, type(d).__contains__ rather than d.__contains__, etc. And to be even
> more pedantic, that's only true for new-style classes.]
>
>
>> I do not see any requirement to call x.__eq__ any particular number of
>> times. The implementation of d might always call somekey.__eq__(x). The
>> concept of sets (and dicts) requires coherent equality comparisons.
>
> To what extent does Python the language specify that user-defined
> classes must be coherent? How much latitude to shoot oneself in the foot
> should the language allow?
>
> What counts as coherent can depend on the types involved. For instance,
> I consider IEEE-754 Not-A-Numbers to be coherent, albeit weird. Python
> goes only so far to accomodate NANs: while it allows a NAN to test
> unequal even to itself (`NAN == NAN` returns False), containers are
> allowed to assume that instances are equal to themselves (`NAN in {NAN}`
> returns True). This was discussed in great detail a few years ago, and
> if I recall correctly, the conclusion was that containers can assume
> that their elements are reflexive (they equal themselves), but equality
> == cannot make the same assumption and bypass calling __eq__.
>
>
>> >where d is a dict would result in only one dict lookup (the second one
>> >being constant folded away). The question is whether it's ok to do it,
>> >despite the fact that it changes the semantics on how many times
>> >__eq__ is called on x.
>>
>> A __eq__ that has side-effects violates the intended and expected
>> semanitics of __eq__.
>
> Nevertheless, an __eq__ with side-effects is legal Python and may in
> fact be useful.
>
> It's a tricky one... I don't know how I feel about short-cutting normal
> Python semantics for speed. On the one hand, faster is good. But on the
> other hand, it makes it harder to reason about code when things go
> wrong. "Why is my __eq__ method not being called?"
>
>
> --
> Steven
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com
note that this is specifically about dicts, where __eq__ will be
called undecided number of times anyway (depending on collisions in
hash/buckets which is implementation specific to start with)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 01:21:05PM +0200, Maciej Fijalkowski wrote: > note that this is specifically about dicts, where __eq__ will be > called undecided number of times anyway (depending on collisions in > hash/buckets which is implementation specific to start with) Exactly. Using a __eq__ method with side-effects is a good way to find out how many collisions your dict has :-) But specifically with your example, if x in d: return d[x] my sense of this is that it falls into the same conceptual area as the identity optimization for checking list or set containment: slightly unclean, but justified. Provided d is an actual built-in dict, and it hasn't been modified between one call and the next, I think it would be okay to optimize the second lookup d[x]. A question: how far away will this optimization apply? if x in d: do_this() do_that() do_something_else() spam = d[x] Assuming no modifications to d, will the second lookup still be optimized? -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
Hello, 2014-03-18 18:13 GMT+09:00 Victor Stinner : > I'm not against backporting the change in Python 3.4.1. It can be seen > as a bugfix. I don't think that anyone wants a Unicode error when > reading or printing non-ASCII data from stdin/to stdout. But I would > like the opinion of other developers before doing that. FYI: Guido was opposed to change error handler of stdin and stdout years ago. http://bugs.python.org/issue2630#msg65493 > Amaury: I think it would be okay to use backslashreplace as the default > error handler for sys.stderr. Probably not for sys.stdout or other > files, since I'm sure many users prefer the errors when their data > cannot be printed rather than silently writing \u escapes that might > cause other code reading their output to choke. For sys.stderr though I > think not having exceptions raised when attempting to print errors is > very valuable. -- Atsuo Ishimoto Mail: [email protected] Twitter: atsuoishimoto ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Call for Python Developers for our humanoid Robot NAO
Hi, We are the manufacturer of the humanoid robot NAO : http://www.youtube.com/watch?v=nNbj2G3GmAo We are now offering great opportunities for developers to use NAO and would like to get in touch with you and your members : http://www.youtube.com/watch?v=_AxErdP0YI8 Let me know if you may be interested. Thank you! Xavier -- Xavier Salort Area Sales Manager M + 1 857 247 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
Hello Xavier, It is not obvious your message is appropriate for python-dev. It looks like mere advertising; if it is not, please explain. To clarify what this mailing-list is about: "On this list the key Python developers discuss the future of the language and its implementation. Topics include Python design issues, release mechanics, and maintenance of existing releases." (from https://mail.python.org/mailman/listinfo/python-dev) Regards Antoine. Le 17/03/2014 23:05, Xavier Salort a écrit : Hi, We are the manufacturer of the humanoid robot NAO : http://www.youtube.com/watch?v=nNbj2G3GmAo We are now offering great opportunities for developers to use NAO and would like to get in touch with you and your members : http://www.youtube.com/watch?v=_AxErdP0YI8 Let me know if you may be interested. Thank you! Xavier -- Xavier Salort Area Sales Manager M + 1 857 247 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
On 18/03/14 16:44, Antoine Pitrou wrote: Hello Xavier, It is not obvious your message is appropriate for python-dev. It looks like mere advertising; if it is not, please explain. To clarify what this mailing-list is about: "On this list the key Python developers discuss the future of the language and its implementation. Topics include Python design issues, release mechanics, and maintenance of existing releases." (from https://mail.python.org/mailman/listinfo/python-dev) Unless you're offering all the core-devs free robots. In which case it's fine. Michael Regards Antoine. Le 17/03/2014 23:05, Xavier Salort a écrit : Hi, We are the manufacturer of the humanoid robot NAO : http://www.youtube.com/watch?v=nNbj2G3GmAo We are now offering great opportunities for developers to use NAO and would like to get in touch with you and your members : http://www.youtube.com/watch?v=_AxErdP0YI8 Let me know if you may be interested. Thank you! Xavier -- Xavier Salort Area Sales Manager M + 1 857 247 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] hg branching + log question
Le 18/03/2014 04:05, Jeff Allen a écrit : I found it helpful to practice extensively on something that doesn't matter. The gap for me is still examples of what I want "done well". Clearly the Python repos represent complex work, but even accepting it is all done well, are without much commentary. This is very good: http://hgbook.red-bean.com/read/ , but there are hints it has not kept up. This also: http://legacy.python.org/dev/peps/pep-0385/ Does the developer’s guide help? http://docs.python.org/devguide/patch.html http://docs.python.org/devguide/faq.html#version-control Regards ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
On 18.03.2014 17:46, Michael Foord wrote: > > On 18/03/14 16:44, Antoine Pitrou wrote: >> >> Hello Xavier, >> >> It is not obvious your message is appropriate for python-dev. It looks >> like mere advertising; if it is not, please explain. >> >> To clarify what this mailing-list is about: "On this list the key >> Python developers discuss the future of the language and its >> implementation. Topics include Python design issues, release >> mechanics, and maintenance of existing releases." >> >> (from https://mail.python.org/mailman/listinfo/python-dev) >> > > Unless you're offering all the core-devs free robots. In which case it's > fine. Sounds like a good deal to me. :) Can the NAO bot do The Silly Walk (tm), too? I'm even willing to film and upload movies of NAO vs. my cats... :) Christian ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
26.02.14 11:40, Serhiy Storchaka написав(ла): Let's choose the least confusing names. See discussions at: http://bugs.python.org/issue3081 http://bugs.python.org/issue16447 http://bugs.python.org/issue20440 http://comments.gmane.org/gmane.comp.python.devel/145346 Poll results: Py_(X)SETREF: +3 (Antoine, Kristján, Nick) Py_(X)DECREC_REPLACE:+3 (Ryan, Georg, Larry) -2 (Antoine, Kristján) Py_(X)ASSIGN, Py_REF_ASSIGN, Py_(X)REPLACE, Py_(X)STORE, Py_SET_AND_(X)DECREF, Py_(X)DECREF_AND_ASSIGN, Py_ASSIGN_AND_(X)DECREF: -1 (Antoine or Kristján) Py_CLEAR_AND_SET: -2 (Antoine, Kristján) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
On Mar 18, 2014, at 08:29 PM, Serhiy Storchaka wrote: >Poll results: > >Py_(X)SETREF: +3 (Antoine, Kristján, Nick) +1 -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
On Tue, 18 Mar 2014 19:21:52 +0100 Christian Heimes wrote: > > Sounds like a good deal to me. :) Can the NAO bot do The Silly Walk > (tm), too? I'm even willing to film and upload movies of NAO vs. my > cats... :) I've heard it can mend a buildbot with its own hands. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
Am 18.03.2014 19:29, schrieb Serhiy Storchaka: > 26.02.14 11:40, Serhiy Storchaka написав(ла): >> Let's choose the least confusing names. >> >> See discussions at: >> >> http://bugs.python.org/issue3081 >> http://bugs.python.org/issue16447 >> http://bugs.python.org/issue20440 >> http://comments.gmane.org/gmane.comp.python.devel/145346 >> > > Poll results: > > Py_(X)SETREF: +3 (Antoine, Kristján, Nick) Well, now that I know that -1 are counted, -1 to this. > Py_(X)DECREC_REPLACE:+3 (Ryan, Georg, Larry) -2 (Antoine, Kristján) > > Py_(X)ASSIGN, Py_REF_ASSIGN, Py_(X)REPLACE, Py_(X)STORE, > Py_SET_AND_(X)DECREF, Py_(X)DECREF_AND_ASSIGN, Py_ASSIGN_AND_(X)DECREF: > -1 (Antoine or Kristján) > > Py_CLEAR_AND_SET: -2 (Antoine, Kristján) Georg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
On 03/18/2014 12:05 PM, Georg Brandl wrote: Am 18.03.2014 19:29, schrieb Serhiy Storchaka: 26.02.14 11:40, Serhiy Storchaka написав(ла): Let's choose the least confusing names. See discussions at: http://bugs.python.org/issue3081 http://bugs.python.org/issue16447 http://bugs.python.org/issue20440 http://comments.gmane.org/gmane.comp.python.devel/145346 Poll results: Py_(X)SETREF: +3 (Antoine, Kristján, Nick) Well, now that I know that -1 are counted, -1 to this. Me too, -1 to Py_SETREF / Py_XSETREF. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/18/2014 07:18 AM, Steven D'Aprano wrote: > Nevertheless, an __eq__ with side-effects is legal Python and may in > fact be useful. E.g., for an LRU usecase. Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlMoobQACgkQ+gerLs4ltQ62agCaA6Z++yLV6VkJ5VUu8Q38AP2N TfcAnjWyYE6/taksZQ0J3igJfv9+KhiK =/D19 -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 4:21 PM, Steven D'Aprano wrote: > On Tue, Mar 18, 2014 at 01:21:05PM +0200, Maciej Fijalkowski wrote: > >> note that this is specifically about dicts, where __eq__ will be >> called undecided number of times anyway (depending on collisions in >> hash/buckets which is implementation specific to start with) > > Exactly. Using a __eq__ method with side-effects is a good way to find > out how many collisions your dict has :-) > > But specifically with your example, > > if x in d: > return d[x] > > my sense of this is that it falls into the same conceptual area as the > identity optimization for checking list or set containment: slightly > unclean, but justified. Provided d is an actual built-in dict, and it > hasn't been modified between one call and the next, I think it would be > okay to optimize the second lookup d[x]. > > A question: how far away will this optimization apply? > > if x in d: > do_this() > do_that() > do_something_else() > spam = d[x] it depends what those functions do. JIT will inline them and if they're small, it should work (although a modification of a different dict is illegal, since aliasing is not proven), but at some point it'll give up (Note that it'll also give up with a call to C releasing GIL since some other thread can modify it). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
On 3/18/2014 12:44 PM, Antoine Pitrou wrote: Hello Xavier, It is not obvious your message is appropriate for python-dev. It looks like mere advertising; The wording is typical for slightly indirect sales pitches in English. The same non-specific (no language mentioned) message was sent to python-list and, I presume, to tens or even hundreds of other lists (with language X substituted for 'Python' in the subject). For python-list, it was caught for moderation and I discarded it. Le 17/03/2014 23:05, Xavier Salort a écrit : Hi, We are the manufacturer of the humanoid robot NAO : http://www.youtube.com/watch?v=nNbj2G3GmAo We are now offering great opportunities for developers to use NAO and would like to get in touch with you and your members : http://www.youtube.com/watch?v=_AxErdP0YI8 Let me know if you may be interested. Thank you! Xavier -- Xavier Salort Area Sales Manager M + 1 857 247 -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
On 19 Mar 2014 05:10, "Larry Hastings" wrote: > > On 03/18/2014 12:05 PM, Georg Brandl wrote: >> >> Am 18.03.2014 19:29, schrieb Serhiy Storchaka: >>> >>> 26.02.14 11:40, Serhiy Storchaka написав(ла): Let's choose the least confusing names. See discussions at: http://bugs.python.org/issue3081 http://bugs.python.org/issue16447 http://bugs.python.org/issue20440 http://comments.gmane.org/gmane.comp.python.devel/145346 >>> >>> Poll results: >>> >>> Py_(X)SETREF: +3 (Antoine, Kristján, Nick) >> >> Well, now that I know that -1 are counted, -1 to this. > > > Me too, -1 to Py_SETREF / Py_XSETREF. I think we're going to have to resort to BDFL appeal on this one - consensus seems unlikely at this point. Cheers, Nick. > > > /arry > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 18 March 2014 19:46, Maciej Fijalkowski wrote: >> A question: how far away will this optimization apply? >> >> if x in d: >> do_this() >> do_that() >> do_something_else() >> spam = d[x] > > it depends what those functions do. JIT will inline them and if > they're small, it should work (although a modification of a different > dict is illegal, since aliasing is not proven), but at some point > it'll give up (Note that it'll also give up with a call to C releasing > GIL since some other thread can modify it). Surely in the presence of threads the optimisation is invalid anyway as other threads can run in between each opcode (I don't know how you'd phrase that in a way that wasn't language dependent other than "everywhere" :-)) so if x in d: # HERE spam = d[x] d can be modified at HERE. (If d is a local variable, obviously the chance that another thread has access to d is a lot lower, but do you really do that level of alias tracking?) Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
Christian Heimes wrote: Can the NAO bot do The Silly Walk (tm), too? From what I gather, making robots do *non*-silly walks is the hard part. -- Greg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
26.02.14 11:40, Serhiy Storchaka написав(ла): Let's choose the least confusing names. See discussions at: http://bugs.python.org/issue3081 http://bugs.python.org/issue16447 http://bugs.python.org/issue20440 http://comments.gmane.org/gmane.comp.python.devel/145346 Updated poll results. There are two leaders: Py_(X)SETREF (originally proposed by Antoine in issue3081): +4 (Antoine, Kristján, Nick, Barry) -2 (Georg, Larry) Py_(X)DECREC_REPLACE (originally proposed by Victor in issue16447): +3 (Ryan, Georg, Larry) -2 (Antoine, Kristján) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
On 2014-03-18 21:06, Nick Coghlan wrote: On 19 Mar 2014 05:10, "Larry Hastings" mailto:[email protected]>> wrote: > > On 03/18/2014 12:05 PM, Georg Brandl wrote: >> >> Am 18.03.2014 19:29, schrieb Serhiy Storchaka: >>> >>> 26.02.14 11:40, Serhiy Storchaka написав(ла): Let's choose the least confusing names. See discussions at: http://bugs.python.org/issue3081 http://bugs.python.org/issue16447 http://bugs.python.org/issue20440 http://comments.gmane.org/gmane.comp.python.devel/145346 >>> >>> Poll results: >>> >>> Py_(X)SETREF: +3 (Antoine, Kristján, Nick) >> >> Well, now that I know that -1 are counted, -1 to this. > > > Me too, -1 to Py_SETREF / Py_XSETREF. I think we're going to have to resort to BDFL appeal on this one - consensus seems unlikely at this point. FWIW, I haven't been following the discussion, but, after a (very) quick look, to me: Py_XDECREF(ptr); ptr = NULL; would be clearer as: Py_CLEAR_REF(ptr); and: Py_XDECREF(ptr); ptr = new_value; would be clearer as: Py_REPLACE_REF(ptr, new_value); ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Poll: Py_REPLACE/Py_ASSIGN/etc
On 19 Mar 2014 07:34, "MRAB" wrote: > FWIW, I haven't been following the discussion, Note that this about correctness, not just clarity - using DECREF on member attributes is not safe, as you may end up exposing a partially destroyed object to other code. > but, after a (very) > quick look, to me: > > Py_XDECREF(ptr); > ptr = NULL; > > would be clearer as: > > Py_CLEAR_REF(ptr); Already exists as Py_CLEAR (with the correct temp variable usage). > and: > > Py_XDECREF(ptr); > ptr = new_value; > > would be clearer as: > > Py_REPLACE_REF(ptr, new_value); That is indeed the one we're aiming to find a suitable name for. Cheers, Nick. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Call for Python Developers for our humanoid Robot NAO
On Tue, Mar 18, 2014 at 1:47 PM, Terry Reedy wrote: > The wording is typical for slightly indirect sales pitches in English. > yeah, it's spam. > The same non-specific (no language mentioned) message was sent to > python-list and, I presume, to tens or even hundreds of other lists (with > language X substituted for 'Python' in the subject). For what it's worth, in the promotional video, they have folks writing python code -- so I think this is in fact Python-relevant. That doesn't make it appropriate for this list, but I think a programmable-in-python robot is pretty cool, in any case. Unless they have different versions of that video with different programming langues edited in -- that would be quite impressive, though! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception [email protected] ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
My 2 cents: it feels like a slippery slope to start guaranteeing the number and ordering of calls to comparison functions -- for instance, doing that for the sort() function would lock in the sort implementation. It feels like the number/ordering of the calls should be "implementation-defined" in the same way that dict iteration order is, or comparisons between incomparable types; I don't think all of CPython's behavior should be guaranteed as part of the language semantics, and that this kind of change wouldn't have to necessarily represent "changing the semantics" if the semantics weren't considered guaranteed in the first place. On the other hand, even if it's theoretically not guaranteed that the sort() function calls __lt__specifically, in practice it'd be a bad idea for an alternative implementation to call __gt__ instead, so I guess there's some aspect of judgment as to what parts should be allowed to be CPython-defined (though my personal take is that this doesn't apply to this particular case). On Tue, Mar 18, 2014 at 7:21 AM, Steven D'Aprano wrote: > On Tue, Mar 18, 2014 at 01:21:05PM +0200, Maciej Fijalkowski wrote: > > > note that this is specifically about dicts, where __eq__ will be > > called undecided number of times anyway (depending on collisions in > > hash/buckets which is implementation specific to start with) > > Exactly. Using a __eq__ method with side-effects is a good way to find > out how many collisions your dict has :-) > > But specifically with your example, > > if x in d: > return d[x] > > my sense of this is that it falls into the same conceptual area as the > identity optimization for checking list or set containment: slightly > unclean, but justified. Provided d is an actual built-in dict, and it > hasn't been modified between one call and the next, I think it would be > okay to optimize the second lookup d[x]. > > A question: how far away will this optimization apply? > > if x in d: > do_this() > do_that() > do_something_else() > spam = d[x] > > Assuming no modifications to d, will the second lookup still be > optimized? > > > > -- > Steven > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/kmod%40dropbox.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On Tue, Mar 18, 2014 at 04:42:29PM -0700, Kevin Modzelewski wrote: > My 2 cents: it feels like a slippery slope to start guaranteeing the number > and ordering of calls to comparison functions -- for instance, doing that > for the sort() function would lock in the sort implementation. Although I agree with your conclusion, I'm not so sure I agree with the way you reach that conclusion. (Actually, I'm not even sure I agree with my own reasoning!) The problem here isn't that Maciej wants to change the implementation of some method or function, like sort. The problem is that (as I understand it) Maciej wants a blessing to change the semantics of multiple Python statements. Currently, the code: if key in dict: return dict[key] performs two dictionary lookups. If you read the code, you can see the two lookups: "key in dict" performs a lookup, and "dict[key]" performs a lookup. Sorry to belabour the obvious, but this gets to the core of the matter. Often this won't matter, but each lookup involves a call to __hash__ and some variable number of calls to __eq__. (Again, as I understand it) Maciej wants to optimize away the second lookup, so that even if you write code like the above, what actually gets executed (modulo guards for modifications to dicts, multiple threads running, etc.) is very different, closer to this in semantics: try: _temp = dict[key] except KeyError: pass else: return _temp I'm not suggesting that PyPy actually will translate the code exactly like this, only that this will be the semantics. The critical point here is that in the Python code you write, there are two *separate* lookups, but in the code that is actually executed, there is only one. Maciej, is my analysis of what you are doing correct? Although I have tentatively said I think this is okay, it is a change in actual semantics of Python code: what you write is no longer what gets run. That makes this *very* different from changing the implementation of sort -- by analogy, its more like changing the semantics of a = f(x) + f(x) to only call f(x) once. I don't think you would call that an implementation detail, would you? Even if justified -- f(x) is a pure, deterministic function with no side-effects -- it would still be a change to the high-level behaviour of the code. Since this proposal is limited only to built-in dicts in scenarios where they cannot be modified between the two lookups, I think that it will be okay. There's no language guarantee as to the number of times that __eq__ will be called (although I would be surprised if __hash__ isn't called twice). But I worry that I have missed something. -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 3/18/2014 9:09 PM, Steven D'Aprano wrote: Currently, the code: if key in dict: return dict[key] performs two dictionary lookups. If you read the code, you can see the two lookups: "key in dict" performs a lookup, and "dict[key]" performs a lookup. The doc says that @deco def f(): pass is equivalent to def f(): pass f = deco(f) That clearly implies two assignments to the containing namespace. But CPython only does one. It actually does something like def (): pass f = deco() This can be detected by defining deco to have the side-effect of checking the local namespace for a binding to 'f' (actually, function reference>.__name__) and reporting via print() before returning. When someone did detect the difference and filed a bug report, Guido declared that the difference did not matter and that the current behavior is close enough to meeting the as-if rule. Optimizing away a dict lookup strikes me as similar to optimizing away a namespace binding, which is typically a dict setting, which would involve a dict lookup. In fact, I imagine that your test class with a __eq__ method that reports calls would be an alternate way to detect the @deco optimization. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issues about relative& absolute import way for Porting from python2.4 to python2.7
Dear,
I just wrote a sample like this:
testPy/
__init__.py
client.py
SoamFactory.c
SoamFactory.so
soamapi.py
sample/testP.py
export PYTHONPATH=$(TEST_LOCATION):$(TEST_LOCATION)/testPy
Here's the source codes:
__init__.py:
import client
client.py
import soamapi
class Client(soamapi.SessionCallback):
def __init__(self):
print "class Client"
super(Client, self).__init__()
soamapi.initialize()
def create_session(self):
sec_cb = soamapi.DefaultSecurityCallback()
self.connection = soamapi.connect(sec_cb)
soamapi.py
import SoamFactory
class ConnectionSecurityCallback(object):
def __init__(self):
print "class ConnectionSecurityCallback"
def __del__(self):
pass
def test_P(self):
print "test_P in class ConnectionSecurityCallback"
class DefaultSecurityCallback(ConnectionSecurityCallback):
def __init__(self):
super(DefaultSecurityCallback, self).__init__()
print "class DefaultSecurityCallback"
def __del__(self):
super(DefaultSecurityCallback, self).__del__()
def test(self):
print "test in class DefaultSecurityCallback"
class SessionCallback(object):
def __init__(self):
pass
def __del__(self):
pass
def connect(security_callback):
return SoamFactory.SoamFactory_connect(security_callback)
def initialize():
SoamFactory.SoamFactory_initialize()
print "call soamapi"
SoamFactory.c
#include "Python.h"
#include "PythonLoop.c"
PyObject* SOAM_API_MODULE = NULL;
PyObject* pyModule = NULL;
static PyObject* SoamFactory_initialize(PyObject* self, PyObject* args){
PyEval_InitThreads();
init();
Py_RETURN_NONE;
}
static PyObject* SoamFactory_connect(PyObject* self, PyObject* args){
PyObject* pySecCallback = NULL;
int ok = PyArg_ParseTuple(args, "O", &pySecCallback);
if (!ok){
printf("parse tuple error!\n");
Py_RETURN_NONE;
}
if (! PyObject_IsInstance(pySecCallback, connectSecCallback)){
printf("pySecCallback is not an instance of
ConnectionSecurityCallback!\n");
Py_RETURN_NONE;
}
printf("Successful!\n");
Py_RETURN_NONE;
}
static PyMethodDef SoamFactory[] = {
{"SoamFactory_connect", SoamFactory_connect, METH_VARARGS, "connection
function"},
{"SoamFactory_initialize", SoamFactory_initialize, METH_VARARGS,
"SoamFactory_initialize"},
{NULL, NULL}
};
void initSoamFactory(){
PyEval_InitThreads();
Py_Initialize();
pyModule = Py_InitModule("SoamFactory", SoamFactory);
SOAM_API_MODULE = PyImport_ImportModule("soamapi");
}
sample/testP.py
import testPy
print ""
submitter = testPy.client.Client()
submitter.create_session()
print ""
When I ran it on python2.4, it worked well, and the result was
call soamapi
after import soamapi--client.py
class Client
class ConnectionSecurityCallback
class DefaultSecurityCallback
Successful!
But when I ran it on python2.7, it worked beyond my expectation, the result was
call soamapi
call soamapi
class Client
class ConnectionSecurityCallback
class DefaultSecurityCallback
pySecCallback is not an instance of ConnectionSecurityCallback!
I found that soamapi was imported twice, and I investigated this is related to
absolute&relative import way. PyImport_ImportModule in python2.7 uses absolute
import way, it will look up sys.path to get soamapi module, and when testP.py
file import testPy module, it will find local module soamapi under testPy
package, and binds module's name to package, as testPy.soamapi.
There are two ways to correct it for python2.7, 1) Don't use import testPy, use
import client directly to avoid using relative; 2) Use from __future__ import
absolute_import to enable absolute import feature.
But there are two Pre-conditions:
1) Should not modify testP.py;
2) Should be ran on both python2.4 and 2.7.
I don't know how to fix it. Is there any official way about how to porting this
scenario or better idea?
Thanks,
Vatel___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
I think in this case, though, if we say for the sake of argument that the guaranteed semantics of a dictionary lookup are zero or more calls to __hash__ plus zero or more calls to __eq__, then two back-to-back dictionary lookups wouldn't have any observable differences from doing only one, unless you start to make assumptions about the behavior of the implementation. To me there seems to be a bit of a gap between seeing a dictionary lookup and knowing the exact sequence of user-functions that get called, far more than for example something like "a < b". I would feel differently if the question was if it's ok to fold something like x = a < b y = a < b into a single comparison, since I'd agree with the way you described it that you look at this code and would expect __lt__ to be called twice. I guess maybe I just disagree about whether dictionaries are contractually bound to call __hash__ and __eq__? For instance, maybe the dict could have a small cache of recently-looked-up elements to skip hashing / equality tests if they get accessed again; I have no idea if that would be profitable or not, but it seems like that would be a roughly-equivalent change that's still "doing two dictionary lookups" except the second one simply wouldn't call back into the user's python code. Or maybe the dictionary could be implemented as a simple list for small sizes and skip calling __hash__ until it decides to switch to a hash-table strategy; again I'd still say it's "doing the lookups" but it just calls a different set of python functions. > Although I have tentatively said I think this is okay, it is a change in > actual semantics of Python code: what you write is no longer what gets > run. That makes this *very* different from changing the implementation > of sort -- by analogy, its more like changing the semantics of > > a = f(x) + f(x) > > to only call f(x) once. I don't think you would call that an > implementation detail, would you? Even if justified -- f(x) is a pure, > deterministic function with no side-effects -- it would still be a > change to the high-level behaviour of the code. > To me, the optimization stage of a compiler's job is to transform a program to an equivalent one that runs faster, where equivalence is defined w.r.t. a certain set of rules defining the behavior of the language. If f() can really be proven to be a function that is deterministic, has no side-effects, does no runtime introspection, and returns a type that supports the identity "x + x == 2 * x" (quite a bit of work for a dynamic language jit, but definitely possible!), then I'd say that I have a fairly different understanding of the "high-level behavior" the runtime is contracted to follow. As a simpler example, I think the runtime should be very free to condense "a = 1 + 1" into "a = 2" without doing the addition. Anyway, as I alluded to about the __lt__ / __gt__ usage in sort(), just because I might want alternative implementations to have flexibility to do something doesn't mean it's reasonable to say it's so. I'm biased since the implementation I'm working on uses std::unordered_map to implement Python dictionaries, and I have no idea how that's actually implemented and I'd rather not have to :) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issues about relative& absolute import way for Porting from python2.4 to python2.7
Dear,
I just wrote a sample like this:
testPy/
__init__.py
client.py
SoamFactory.c
SoamFactory.so
soamapi.py
sample/testP.py
export PYTHONPATH=$(TEST_LOCATION):$(TEST_LOCATION)/testPy
Here's the source codes:
__init__.py:
import client
client.py
import soamapi
class Client(soamapi.SessionCallback):
def __init__(self):
print "class Client"
super(Client, self).__init__()
soamapi.initialize()
def create_session(self):
sec_cb = soamapi.DefaultSecurityCallback()
self.connection = soamapi.connect(sec_cb)
soamapi.py
import SoamFactory
class ConnectionSecurityCallback(object):
def __init__(self):
print "class ConnectionSecurityCallback"
def __del__(self):
pass
def test_P(self):
print "test_P in class ConnectionSecurityCallback"
class DefaultSecurityCallback(ConnectionSecurityCallback):
def __init__(self):
super(DefaultSecurityCallback, self).__init__()
print "class DefaultSecurityCallback"
def __del__(self):
super(DefaultSecurityCallback, self).__del__()
def test(self):
print "test in class DefaultSecurityCallback"
class SessionCallback(object):
def __init__(self):
pass
def __del__(self):
pass
def connect(security_callback):
return SoamFactory.SoamFactory_connect(security_callback)
def initialize():
SoamFactory.SoamFactory_initialize()
print "call soamapi"
SoamFactory.c
#include "Python.h"
#include "PythonLoop.c"
PyObject* SOAM_API_MODULE = NULL;
PyObject* pyModule = NULL;
static PyObject* SoamFactory_initialize(PyObject* self, PyObject* args){
PyEval_InitThreads();
init();
Py_RETURN_NONE;
}
static PyObject* SoamFactory_connect(PyObject* self, PyObject* args){
PyObject* pySecCallback = NULL;
int ok = PyArg_ParseTuple(args, "O", &pySecCallback);
if (!ok){
printf("parse tuple error!\n");
Py_RETURN_NONE;
}
if (! PyObject_IsInstance(pySecCallback, connectSecCallback)){
printf("pySecCallback is not an instance of
ConnectionSecurityCallback!\n");
Py_RETURN_NONE;
}
printf("Successful!\n");
Py_RETURN_NONE;
}
static PyMethodDef SoamFactory[] = {
{"SoamFactory_connect", SoamFactory_connect, METH_VARARGS, "connection
function"},
{"SoamFactory_initialize", SoamFactory_initialize, METH_VARARGS,
"SoamFactory_initialize"},
{NULL, NULL}
};
void initSoamFactory(){
PyEval_InitThreads();
Py_Initialize();
pyModule = Py_InitModule("SoamFactory", SoamFactory);
SOAM_API_MODULE = PyImport_ImportModule("soamapi");
}
sample/testP.py
import testPy
print ""
submitter = testPy.client.Client()
submitter.create_session()
print ""
When I ran it on python2.4, it worked well, and the result was
call soamapi
after import soamapi--client.py
class Client
class ConnectionSecurityCallback
class DefaultSecurityCallback
Successful!
But when I ran it on python2.7, it worked beyond my expectation, the result was
call soamapi
call soamapi
class Client
class ConnectionSecurityCallback
class DefaultSecurityCallback
pySecCallback is not an instance of ConnectionSecurityCallback!
I found that soamapi was imported twice, and I investigated this is related to
absolute&relative import way. PyImport_ImportModule in python2.7 uses absolute
import way, it will look up sys.path to get soamapi module, and when testP.py
file import testPy module, it will find local module soamapi under testPy
package, and binds module's name to package, as testPy.soamapi.
There are two ways to correct it for python2.7, 1) Don't use import testPy, use
import client directly to avoid using relative; 2) Use from __future__ import
absolute_import to enable absolute import feature.
But there are two Pre-conditions:
1) Should not modify testP.py;
2) Should be ran on both python2.4 and 2.7.
I don't know how to fix it. Is there any official way about how to porting this
scenario or better idea?
Thanks,
Vatel___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
On 19 March 2014 11:09, Steven D'Aprano wrote: > Although I have tentatively said I think this is okay, it is a change in > actual semantics of Python code: what you write is no longer what gets > run. That makes this *very* different from changing the implementation > of sort -- by analogy, its more like changing the semantics of > > a = f(x) + f(x) > > to only call f(x) once. I don't think you would call that an > implementation detail, would you? Even if justified -- f(x) is a pure, > deterministic function with no side-effects -- it would still be a > change to the high-level behaviour of the code. Ah, I think this is a good alternative example. Given the stated assumptions (true builtin dict, not modified between operations), would we be OK with PyPI optimising the following to only do a single dict lookup: a = d[x] + d[x] It's essentially the same optimisation as the one being discussed - in the code *as written*, there are two lookups visible, but for any sensible __hash__ and __eq__ implementation, they should always give the same answer for a true builtin dict that isn't modified between the two lookups. (and yes, PyPy has the infrastructure to enforce those constraints safely and fall back to normal execution if they aren't met - that ability to take opportunistic advantage of known behaviours of particular types is one of the key things that makes it such a powerful tool for implicit optimisations, as compared to things like Cython and Numba, which change semantics more markedly, but also have to be explicitly applied to specific sections of your code rather than being applied automatically). I think it's certainly borderline (it's the kind of surprising behavioural change that irritates people about C/C++ optimisers), but I also think it's a defensible optimisation if the gain is significant enough. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.5 now uses surrogateescape for the POSIX locale
Victor Stinner writes: > 2014-03-18 11:02 GMT+01:00 Atsuo Ishimoto : > > FYI: Guido was opposed to change error handler of stdin and > > stdout years ago. > > > > http://bugs.python.org/issue2630#msg65493 > > This issue proposes to use "backslashreplace" error handler for > stdout. This error handler is very different to "surrogateescape" > which is related to PEP 383 and used by all OS functions. I would say, it's not different in the relevant aspect, which is spewing presumably unreadable bytes that may cause other code reading the output to choke. I would think backslashreplace would generally be preferred for these use cases. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intricacies of calling __eq__
Kevin Modzelewski writes: > I think in this case, though, if we say for the sake of argument > that the guaranteed semantics of a dictionary lookup are zero or I don't understand the point of that argument. It's simply false that semantics are guaranteed, and all of the dunders might be user functions. > more calls to __hash__ plus zero or more calls to __eq__, then two > back-to-back dictionary lookups wouldn't have any observable > differences from doing only one, unless you start to make > assumptions about the behavior of the implementation. That's false. The inverse is true: you should allow the possibility of observable differences, unless you make assumptions about the behavior (implying there are none). > To me there seems to be a bit of a gap between seeing a dictionary > lookup and knowing the exact sequence of user-functions that get > called, far more than for example something like "a < b". The point here is that we *know* that there may be a user function (the dunder that implements []) being called, and it is very hard to determine that that function is pure. Your example of a caching hash is exactly the kind of impure function that one would expect, but who knows what might be called -- there could be a reference to a database on Mars involved (do we have a vehicle on Mars at the moment? anyway...), which calls a pile of Twisted code, and has latencies of many seconds. So Steven is precisely right -- in order to allow this optimization, it would have to be explicitly allowed. Like Steven, I have no strong feeling against it, but then, I don't have a program talking to a deep space vehicle in my near future. Darn it! :-( ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
