Re: [Python-Dev] Mac popups running make test
> On 12 May 2015, at 18:14, Tal Einat wrote: > > On Tue, May 12, 2015 at 4:14 PM, Skip Montanaro > wrote: >> >>> Twice now, I've gotten this popup: ... >> >> Let me improve my request, as it seems there is some confusion about >> what I want. I'm specifically not asking that the popups not be >> displayed. I don't mind dismissing them. When they appear, I would, >> however, like to glance over at the stream of messages emitted by the >> test runner and see a message about it being expected. It seems that >> the tests which can trigger the crash reporter do this. > > In my case, the popups appear but then disappear within a fraction of > a second, and this happens about 10-20 times when running the full > test suite. So I don't have a chance to interact with the popups, and > this causes test failures. > > Also, when running a large suite of tests, I may not be looking at the > screen by the time these popups appear. I wouldn't want the tests to > fail nor would I want the test run to stall. > > I can't test this right now, but does disabling the "network" resource > avoid these popups? Though even if it does we'll still need a way to > run network-related tests on OSX. The only way I know to easily avoid these pop-ups is to turn off the local firewall while testing. Signing the interpreter likely also works, but probably only when using a paid developer certificate. Ronald ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How shall we conduct the Python 3.5 beta and rc periods? (Please vote!)
On 05/12/2015 10:04 AM, Larry Hastings wrote: What do you think? [...] Please cast your votes workflow 012 Larry Hastings-0.5 10.5 Brett Cannon 010 Nick Coghlan 010 Chris Angelico 000“in favor of [Workflow 1]” Ned Deily -11 -0.5 Barry Warsaw 000“I'm fine with any of them as long as the workflow is well documented” Antoine Pitrou-0.5 10.5 Ethan Furman 010didn't cast explicit votes for anything but workflow 1 Berker Peksag 010 Yuri Selivanov 010didn't cast explicit votes for anything but workflow 1 Jesus Cea 010 total -290.5 I'm calling it--the winner by a landslide is the ambitious Workflow 1. Barry and Chris A didn't cast any explicit votes, but both approved of Workflow 1. Every other voter was +1 on Workflow 1 and uninterested or negative on the others. I'll start experimenting with the workflow(s) and will add documentation to the Dev Guide. The fun starts next weekend, //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 455 -- TransformDict
Before the Python 3.5 feature freeze, I should step-up and formally reject PEP 455 for "Adding a key-transforming dictionary to collections". I had completed an involved review effort a long time ago and I apologize for the delay in making the pronouncement. What made it a interesting choice from the outset is that the idea of a "transformation" is an enticing concept that seems full of possibility. I spent a good deal of time exploring what could be done with it but found that it mostly fell short of its promise. There were many issues. Here are some that were at the top: * Most use cases don't need or want the reverse lookup feature (what is wanted is a set of one-way canonicalization functions). Those that do would want to have a choice of what is saved (first stored, last stored, n most recent, a set of all inputs, a list of all inputs, nothing, etc). In database terms, it models a many-to-one table (the canonicalization or transformation function) with the one being a primary key into another possibly surjective table of two columns (the key/value store). A surjection into another surjection isn't inherently reversible in a useful way, nor does it seem to be a common way to model data. * People are creative at coming up with using cases for the TD but then find that the resulting code is less clear, slower, less intuitive, more memory intensive, and harder to debug than just using a plain dict with a function call before the lookup: d[func(key)]. It was challenging to find any existing code that would be made better by the availability of the TD. * The TD seems to be all about combining data scrubbing (case-folding, unicode canonicalization, type-folding, object identity, unit-conversion, or finding a canonical member of an equivalence class) with a mapping (looking-up a value for a given key). Those two operations are conceptually orthogonal. The former doesn't get easier when hidden behind a mapping API and the latter loses the flexibility of choosing your preferred mapping (an ordereddict, a persistentdict, a chainmap, etc) and the flexibility of establishing your own rules for whether and how to do a reverse lookup. Raymond Hettinger P.S. Besides the core conceptual issues listed above, there are a number of smaller issues with the TD that surfaced during design review sessions. In no particular order, here are a few of the observations: * It seems to require above average skill to figure-out what can be used as a transform function. It is more expert-friendly than beginner friendly. It takes a little while to get used to it. It wasn't self-evident that transformations happen both when a key is stored and again when it is looked-up (contrast this with key-functions for sorting which are called at most once per key). * The name, TransformDict, suggests that it might transform the value instead of the key or that it might transform the dictionary into something else. The name TransformDict is so general that it would be hard to discover when faced with a specific problem. The name also limits perception of what could be done with it (i.e. a function that logs accesses but doesn't actually change the key). * The tool doesn't self describe itself well. Looking at the help(), or the __repr__(), or the tooltips did not provide much insight or clarity. The dir() shows many of the _abc implementation details rather than the API itself. * The original key is stored and if you change it, the change isn't stored. The _original dict is private (perhaps to reduce the risk of putting the TD in an inconsistent state) but this limits access to the stored data. * The TD is unsuitable for bijections because the API is inherently biased with a rich group of operators and methods for forward lookup but has only one method for reverse lookup. * The reverse feature is hard to find (getitem vs __getitem__) and its output pair is surprising and a bit awkward to use. It provides only one accessor method rather that the full dict API that would be given by a second dictionary. The API hides the fact that there are two underlying dictionaries. * It was surprising that when d[k] failed, it failed with transformation exception rather than a KeyError, violating the expectations of the calling code (for example, if the transformation function is int(), the call d["12"] transforms to d[12] and either succeeds in returning a value or in raising a KeyError, but the call d["12.0"] fails with a TypeError). The latter issue limits its substitutability into existing code that expects real mappings and for exposing to end-users as if it were a normal dictionary. * There were other issues with dict invariants as well and these affected substitutability in a sometimes subtle way. For example, the TD does not work with __missing__(). Also, "k in td" does not imply that "k in list(td.keys
Re: [Python-Dev] PEP 455 -- TransformDict
Thanks for this thorough review, Raymond! Especially the user research is amazing. And thanks for Antoine for writing the PEP -- you never know how an idea pans out until you've tried it. --Guido On Thu, May 14, 2015 at 7:29 AM, Raymond Hettinger < [email protected]> wrote: > Before the Python 3.5 feature freeze, I should step-up and > formally reject PEP 455 for "Adding a key-transforming > dictionary to collections". > > I had completed an involved review effort a long time ago > and I apologize for the delay in making the pronouncement. > > What made it a interesting choice from the outset is that the > idea of a "transformation" is an enticing concept that seems > full of possibility. I spent a good deal of time exploring > what could be done with it but found that it mostly fell short > of its promise. > > There were many issues. Here are some that were at the top: > > * Most use cases don't need or want the reverse lookup feature > (what is wanted is a set of one-way canonicalization functions). > Those that do would want to have a choice of what is saved > (first stored, last stored, n most recent, a set of all inputs, > a list of all inputs, nothing, etc). In database terms, it > models a many-to-one table (the canonicalization or > transformation function) with the one being a primary key into > another possibly surjective table of two columns (the > key/value store). A surjection into another surjection isn't > inherently reversible in a useful way, nor does it seem to be a > common way to model data. > > * People are creative at coming up with using cases for the TD > but then find that the resulting code is less clear, slower, > less intuitive, more memory intensive, and harder to debug than > just using a plain dict with a function call before the lookup: > d[func(key)]. It was challenging to find any existing code > that would be made better by the availability of the TD. > > * The TD seems to be all about combining data scrubbing > (case-folding, unicode canonicalization, type-folding, object > identity, unit-conversion, or finding a canonical member of an > equivalence class) with a mapping (looking-up a value for a > given key). Those two operations are conceptually orthogonal. > The former doesn't get easier when hidden behind a mapping API > and the latter loses the flexibility of choosing your preferred > mapping (an ordereddict, a persistentdict, a chainmap, etc) and > the flexibility of establishing your own rules for whether and > how to do a reverse lookup. > > > Raymond Hettinger > > > P.S. Besides the core conceptual issues listed above, there > are a number of smaller issues with the TD that surfaced > during design review sessions. In no particular order, here > are a few of the observations: > > * It seems to require above average skill to figure-out what > can be used as a transform function. It is more > expert-friendly than beginner friendly. It takes a little > while to get used to it. It wasn't self-evident that > transformations happen both when a key is stored and again > when it is looked-up (contrast this with key-functions for > sorting which are called at most once per key). > > * The name, TransformDict, suggests that it might transform the > value instead of the key or that it might transform the > dictionary into something else. The name TransformDict is so > general that it would be hard to discover when faced with a > specific problem. The name also limits perception of what > could be done with it (i.e. a function that logs accesses > but doesn't actually change the key). > > * The tool doesn't self describe itself well. Looking at the > help(), or the __repr__(), or the tooltips did not provide > much insight or clarity. The dir() shows many of the > _abc implementation details rather than the API itself. > > * The original key is stored and if you change it, the change > isn't stored. The _original dict is private (perhaps to > reduce the risk of putting the TD in an inconsistent state) > but this limits access to the stored data. > > * The TD is unsuitable for bijections because the API is > inherently biased with a rich group of operators and methods > for forward lookup but has only one method for reverse lookup. > > * The reverse feature is hard to find (getitem vs __getitem__) > and its output pair is surprising and a bit awkward to use. > It provides only one accessor method rather that the full > dict API that would be given by a second dictionary. The > API hides the fact that there are two underlying dictionaries. > > * It was surprising that when d[k] failed, it failed with > transformation exception rather than a KeyError, violating > the expectations of the calling code (for example, if the > transformation function is int(), the call d["12"] > transforms to d[12] and either succeeds in returning a value > or in
Re: [Python-Dev] PEP 455 -- TransformDict
On 15 May 2015 at 00:41, Guido van Rossum wrote: > Thanks for this thorough review, Raymond! Especially the user research is > amazing. > > And thanks for Antoine for writing the PEP -- you never know how an idea > pans out until you've tried it. Hear, hear! I thought the TransformDict idea sounded interesting when Antoine proposed it, but Raymond's rationale for the rejection makes a great deal of sense. Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python-versus-CPython question for __mul__ dispatch
Hi all, While attempting to clean up some of the more squamous aspects of numpy's operator dispatch code [1][2], I've encountered a situation where the semantics we want and are using are possible according to CPython-the-interpreter, but AFAICT ought not to be possible according to Python-the-language, i.e., it's not clear to me whether it's possible even in principle to implement an object that works the way numpy.ndarray does in any other interpreter. Which makes me a bit nervous, so I wanted to check if there was any ruling on this. Specifically, the quirk we are relying on is this: in CPython, if you do [1, 2] * my_object then my_object's __rmul__ gets called *before* list.__mul__, *regardless* of the inheritance relationship between list and type(my_object). This occurs as a side-effect of the weirdness involved in having both tp_as_number->nb_multiply and tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b", CPython tries a's nb_multiply, then b's nb_multiply, then a's sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an nb_multiply, this means that my_object's nb_multiply gets called before any list method. Here's an example demonstrating how weird this is. list.__mul__ wants an integer, and by "integer" it means "any object with an __index__ method". So here's a class that list is happy to be multiplied by -- according to the ordinary rules for operator dispatch, in the example below Indexable.__mul__ and __rmul__ shouldn't even get a look-in: In [3]: class Indexable(object): ...: def __index__(self): ...: return 2 ...: In [4]: [1, 2] * Indexable() Out[4]: [1, 2, 1, 2] But, if I add an __rmul__ method, then this actually wins: In [6]: class IndexableWithMul(object): ...: def __index__(self): ...: return 2 ...: def __mul__(self, other): ...: return "indexable forward mul" ...: def __rmul__(self, other): ...: return "indexable reverse mul" In [7]: [1, 2] * IndexableWithMul() Out[7]: 'indexable reverse mul' In [8]: IndexableWithMul() * [1, 2] Out[8]: 'indexable forward mul' NumPy arrays, of course, correctly define both __index__ method (which raises an array on general arrays but coerces to int for arrays that contain exactly 1 integer), and also defines an nb_multiply slot which accepts lists and performs elementwise multiplication: In [9]: [1, 2] * np.array(2) Out[9]: array([2, 4]) And that's all great! Just what we want. But the only reason this is possible, AFAICT, is that CPython 'list' is a weird type with undocumented behaviour that you can't actually define using pure Python code. Should I be worried? -n [1] https://github.com/numpy/numpy/pull/5864 [2] https://github.com/numpy/numpy/issues/5844 -- Nathaniel J. Smith -- http://vorpus.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
>>Should I be worried? You mean should *I* be worried ;) Stuff like this is highly relevant for JyNI, so thanks very much for clarifying this subtle behavior. It went onto my todo-list right now to ensure that JyNI will emulate this behavior as soon as I am done with gc-support. (Hopefully it will be feasible, but I can only tell in half a year or so since there are currently other priorities.) Still, this "essay" potentially will save me a lot of time. So, everybody please feel encouraged to post things like this as they come up. Maybe there could be kind of a pitfalls-page somewhere in the docs collecting these things. Best Stefan > Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr > Von: "Nathaniel Smith" > An: "Python Dev" > Betreff: [Python-Dev] Python-versus-CPython question for __mul__ dispatch > > Hi all, > > While attempting to clean up some of the more squamous aspects of > numpy's operator dispatch code [1][2], I've encountered a situation > where the semantics we want and are using are possible according to > CPython-the-interpreter, but AFAICT ought not to be possible according > to Python-the-language, i.e., it's not clear to me whether it's > possible even in principle to implement an object that works the way > numpy.ndarray does in any other interpreter. Which makes me a bit > nervous, so I wanted to check if there was any ruling on this. > > Specifically, the quirk we are relying on is this: in CPython, if you do > > [1, 2] * my_object > > then my_object's __rmul__ gets called *before* list.__mul__, > *regardless* of the inheritance relationship between list and > type(my_object). This occurs as a side-effect of the weirdness > involved in having both tp_as_number->nb_multiply and > tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b", > CPython tries a's nb_multiply, then b's nb_multiply, then a's > sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an > nb_multiply, this means that my_object's nb_multiply gets called > before any list method. > > Here's an example demonstrating how weird this is. list.__mul__ wants > an integer, and by "integer" it means "any object with an __index__ > method". So here's a class that list is happy to be multiplied by -- > according to the ordinary rules for operator dispatch, in the example > below Indexable.__mul__ and __rmul__ shouldn't even get a look-in: > > In [3]: class Indexable(object): >...: def __index__(self): >...: return 2 >...: > > In [4]: [1, 2] * Indexable() > Out[4]: [1, 2, 1, 2] > > But, if I add an __rmul__ method, then this actually wins: > > In [6]: class IndexableWithMul(object): >...: def __index__(self): >...: return 2 > ...: def __mul__(self, other): >...: return "indexable forward mul" >...: def __rmul__(self, other): >...: return "indexable reverse mul" > > In [7]: [1, 2] * IndexableWithMul() > Out[7]: 'indexable reverse mul' > > In [8]: IndexableWithMul() * [1, 2] > Out[8]: 'indexable forward mul' > > NumPy arrays, of course, correctly define both __index__ method (which > raises an array on general arrays but coerces to int for arrays that > contain exactly 1 integer), and also defines an nb_multiply slot which > accepts lists and performs elementwise multiplication: > > In [9]: [1, 2] * np.array(2) > Out[9]: array([2, 4]) > > And that's all great! Just what we want. But the only reason this is > possible, AFAICT, is that CPython 'list' is a weird type with > undocumented behaviour that you can't actually define using pure > Python code. > > Should I be worried? > > -n > > [1] https://github.com/numpy/numpy/pull/5864 > [2] https://github.com/numpy/numpy/issues/5844 > > -- > Nathaniel J. Smith -- http://vorpus.org > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
I expect you can make something that behaves like list by defining __mul__ and __rmul__ and returning NotImplemented. On Thursday, May 14, 2015, Stefan Richthofer wrote: > >>Should I be worried? > > You mean should *I* be worried ;) > > Stuff like this is highly relevant for JyNI, so thanks very much for > clarifying this > subtle behavior. It went onto my todo-list right now to ensure that JyNI > will emulate > this behavior as soon as I am done with gc-support. (Hopefully it will be > feasible, > but I can only tell in half a year or so since there are currently other > priorities.) > Still, this "essay" potentially will save me a lot of time. > > So, everybody please feel encouraged to post things like this as they come > up. Maybe > there could be kind of a pitfalls-page somewhere in the docs collecting > these things. > > Best > > Stefan > > > > Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr > > Von: "Nathaniel Smith" > > > An: "Python Dev" > > > Betreff: [Python-Dev] Python-versus-CPython question for __mul__ dispatch > > > > Hi all, > > > > While attempting to clean up some of the more squamous aspects of > > numpy's operator dispatch code [1][2], I've encountered a situation > > where the semantics we want and are using are possible according to > > CPython-the-interpreter, but AFAICT ought not to be possible according > > to Python-the-language, i.e., it's not clear to me whether it's > > possible even in principle to implement an object that works the way > > numpy.ndarray does in any other interpreter. Which makes me a bit > > nervous, so I wanted to check if there was any ruling on this. > > > > Specifically, the quirk we are relying on is this: in CPython, if you do > > > > [1, 2] * my_object > > > > then my_object's __rmul__ gets called *before* list.__mul__, > > *regardless* of the inheritance relationship between list and > > type(my_object). This occurs as a side-effect of the weirdness > > involved in having both tp_as_number->nb_multiply and > > tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b", > > CPython tries a's nb_multiply, then b's nb_multiply, then a's > > sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an > > nb_multiply, this means that my_object's nb_multiply gets called > > before any list method. > > > > Here's an example demonstrating how weird this is. list.__mul__ wants > > an integer, and by "integer" it means "any object with an __index__ > > method". So here's a class that list is happy to be multiplied by -- > > according to the ordinary rules for operator dispatch, in the example > > below Indexable.__mul__ and __rmul__ shouldn't even get a look-in: > > > > In [3]: class Indexable(object): > >...: def __index__(self): > >...: return 2 > >...: > > > > In [4]: [1, 2] * Indexable() > > Out[4]: [1, 2, 1, 2] > > > > But, if I add an __rmul__ method, then this actually wins: > > > > In [6]: class IndexableWithMul(object): > >...: def __index__(self): > >...: return 2 > > ...: def __mul__(self, other): > >...: return "indexable forward mul" > >...: def __rmul__(self, other): > >...: return "indexable reverse mul" > > > > In [7]: [1, 2] * IndexableWithMul() > > Out[7]: 'indexable reverse mul' > > > > In [8]: IndexableWithMul() * [1, 2] > > Out[8]: 'indexable forward mul' > > > > NumPy arrays, of course, correctly define both __index__ method (which > > raises an array on general arrays but coerces to int for arrays that > > contain exactly 1 integer), and also defines an nb_multiply slot which > > accepts lists and performs elementwise multiplication: > > > > In [9]: [1, 2] * np.array(2) > > Out[9]: array([2, 4]) > > > > And that's all great! Just what we want. But the only reason this is > > possible, AFAICT, is that CPython 'list' is a weird type with > > undocumented behaviour that you can't actually define using pure > > Python code. > > > > Should I be worried? > > > > -n > > > > [1] https://github.com/numpy/numpy/pull/5864 > > [2] https://github.com/numpy/numpy/issues/5844 > > > > -- > > Nathaniel J. Smith -- http://vorpus.org > > ___ > > Python-Dev mailing list > > [email protected] > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (on iPad) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: inspect: Micro-optimize __eq__ for Signature, Parameter and BoundArguments
On 15.05.15 01:23, yury.selivanov wrote: https://hg.python.org/cpython/rev/f0b10980b19e changeset: 96056:f0b10980b19e parent: 96054:15701e89d710 user:Yury Selivanov date:Thu May 14 18:20:01 2015 -0400 summary: inspect: Micro-optimize __eq__ for Signature, Parameter and BoundArguments Provide __ne__ method for consistency. files: Lib/inspect.py | 32 ++-- 1 files changed, 22 insertions(+), 10 deletions(-) diff --git a/Lib/inspect.py b/Lib/inspect.py --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -2353,11 +2353,15 @@ return hash((self.name, self.kind, self.annotation, self.default)) def __eq__(self, other): -return (issubclass(other.__class__, Parameter) and -self._name == other._name and -self._kind == other._kind and -self._default == other._default and -self._annotation == other._annotation) +return (self is other or +(issubclass(other.__class__, Parameter) and + self._name == other._name and + self._kind == other._kind and + self._default == other._default and + self._annotation == other._annotation)) It would be better to return NotImplemented if other is not an instance of Parameter. if self is other: return True if not isinstance(other, Parameter): return NotImplemented return (self._name == other._name and self._kind == other._kind and self._default == other._default and self._annotation == other._annotation) And why you use issubclass() instead of isinstance()? +def __ne__(self, other): +return not self.__eq__(other) This is not need (and incorrect if __eq__ returns NotImplemented). The default __ne__ implementations calls __eq__ and correctly handles NotImplemented. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: inspect: Add __slots__ to BoundArguments.
On 14.05.15 00:38, yury.selivanov wrote: https://hg.python.org/cpython/rev/ee31277386cb changeset: 96038:ee31277386cb user:Yury Selivanov date:Wed May 13 17:18:41 2015 -0400 summary: inspect: Add __slots__ to BoundArguments. Note that adding __slots__ breaks pickleability. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-versus-CPython question for __mul__ dispatch
On Thu, May 14, 2015 at 9:29 PM, Guido van Rossum wrote: > I expect you can make something that behaves like list by defining __mul__ > and __rmul__ and returning NotImplemented. Hmm, it's fairly tricky, and part of the trick is that you can never return NotImplemented (because you have to pretty much take over and entirely replace the normal dispatch rules inside __mul__ and __rmul__), but see attached for something I think should work. So I guess this is just how Python's list, tuple, etc. work, and PyPy and friends need to match... -n > On Thursday, May 14, 2015, Stefan Richthofer > wrote: >> >> >>Should I be worried? >> >> You mean should *I* be worried ;) >> >> Stuff like this is highly relevant for JyNI, so thanks very much for >> clarifying this >> subtle behavior. It went onto my todo-list right now to ensure that JyNI >> will emulate >> this behavior as soon as I am done with gc-support. (Hopefully it will be >> feasible, >> but I can only tell in half a year or so since there are currently other >> priorities.) >> Still, this "essay" potentially will save me a lot of time. >> >> So, everybody please feel encouraged to post things like this as they come >> up. Maybe >> there could be kind of a pitfalls-page somewhere in the docs collecting >> these things. >> >> Best >> >> Stefan >> >> >> > Gesendet: Freitag, 15. Mai 2015 um 02:45 Uhr >> > Von: "Nathaniel Smith" >> > An: "Python Dev" >> > Betreff: [Python-Dev] Python-versus-CPython question for __mul__ >> > dispatch >> > >> > Hi all, >> > >> > While attempting to clean up some of the more squamous aspects of >> > numpy's operator dispatch code [1][2], I've encountered a situation >> > where the semantics we want and are using are possible according to >> > CPython-the-interpreter, but AFAICT ought not to be possible according >> > to Python-the-language, i.e., it's not clear to me whether it's >> > possible even in principle to implement an object that works the way >> > numpy.ndarray does in any other interpreter. Which makes me a bit >> > nervous, so I wanted to check if there was any ruling on this. >> > >> > Specifically, the quirk we are relying on is this: in CPython, if you do >> > >> > [1, 2] * my_object >> > >> > then my_object's __rmul__ gets called *before* list.__mul__, >> > *regardless* of the inheritance relationship between list and >> > type(my_object). This occurs as a side-effect of the weirdness >> > involved in having both tp_as_number->nb_multiply and >> > tp_as_sequence->sq_repeat in the C API -- when evaluating "a * b", >> > CPython tries a's nb_multiply, then b's nb_multiply, then a's >> > sq_repeat, then b's sq_repeat. Since list has an sq_repeat but not an >> > nb_multiply, this means that my_object's nb_multiply gets called >> > before any list method. >> > >> > Here's an example demonstrating how weird this is. list.__mul__ wants >> > an integer, and by "integer" it means "any object with an __index__ >> > method". So here's a class that list is happy to be multiplied by -- >> > according to the ordinary rules for operator dispatch, in the example >> > below Indexable.__mul__ and __rmul__ shouldn't even get a look-in: >> > >> > In [3]: class Indexable(object): >> >...: def __index__(self): >> >...: return 2 >> >...: >> > >> > In [4]: [1, 2] * Indexable() >> > Out[4]: [1, 2, 1, 2] >> > >> > But, if I add an __rmul__ method, then this actually wins: >> > >> > In [6]: class IndexableWithMul(object): >> >...: def __index__(self): >> >...: return 2 >> > ...: def __mul__(self, other): >> >...: return "indexable forward mul" >> >...: def __rmul__(self, other): >> >...: return "indexable reverse mul" >> > >> > In [7]: [1, 2] * IndexableWithMul() >> > Out[7]: 'indexable reverse mul' >> > >> > In [8]: IndexableWithMul() * [1, 2] >> > Out[8]: 'indexable forward mul' >> > >> > NumPy arrays, of course, correctly define both __index__ method (which >> > raises an array on general arrays but coerces to int for arrays that >> > contain exactly 1 integer), and also defines an nb_multiply slot which >> > accepts lists and performs elementwise multiplication: >> > >> > In [9]: [1, 2] * np.array(2) >> > Out[9]: array([2, 4]) >> > >> > And that's all great! Just what we want. But the only reason this is >> > possible, AFAICT, is that CPython 'list' is a weird type with >> > undocumented behaviour that you can't actually define using pure >> > Python code. >> > >> > Should I be worried? >> > >> > -n >> > >> > [1] https://github.com/numpy/numpy/pull/5864 >> > [2] https://github.com/numpy/numpy/issues/5844 >> > >> > -- >> > Nathaniel J. Smith -- http://vorpus.org >> > ___ >> > Python-Dev mailing list >> > [email protected] >> > https://mail.python.org/mailman/listinfo/python-dev >> > Unsubscribe: >> > https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de >> > >>
