[Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Ethan Furman 

Just to be clear, this is about NotImplemented, not NotImplementedError.

tl;dr  When a binary operation fails, should an exception be raised or 
NotImplemented returned?


When a binary operation in Python is attempted, there are two possibilities:

  - it can work
  - it can't work

The main reason [1] that it can't work is that the two operands are of different types, and the first type does not know 
how to deal with the second type.


The question then becomes: how does the first type tell Python that it cannot perform the requested operation?  The most 
obvious answer is to raise an exception, and TypeError is a good candidate.  The problem with the exception raising 
approach is that once an exception is raised, Python doesn't try anything else to make the operation work.


What's wrong with that?  Well, the second type might know how to perform the operation, and in fact that is why we have 
the reflected special methods, such as __radd__ and __rmod__ -- but if the first type raises an exception the __rxxx__ 
methods will not be tried.


Okay, how can the first type tell Python that it cannot do what is requested, but to go ahead and check with the second 
type to see if it does?  That is where NotImplemented comes in -- if a special method (and only a special method) 
returns NotImplemented then Python will check to see if there is anything else it can do to make the operation succeed; 
if all attempts return NotImplemented, then Python itself will raise an appropriate exception [2].


In an effort to see how often NotImplemented is currently being returned I crafted a test script [3] to test the types 
bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, deque, and OrderedDict with the operations for 
__add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, __ilshift__, __imod__, __imul__, __ior__, __ipow__, 
__irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, __mul__, __or__, __pow__, __rshift__, __sub__, 
__truediv__, and __xor__.


Here are the results of the 275 tests:

testing control...

ipow -- Exception  raised
errors in Control -- misunderstanding or bug?

testing types against a foreign class

iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> 
raised instead of TypeError
iand(Counter()) -- NotImplemented not returned, TypeError not raised
ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> 
raised instead of TypeError
isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute 'items'> 
raised instead of TypeError


testing types against a subclass

mod(str()) -- NotImplemented not returned, TypeError not raised

iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised 
(should have worked)
iand(Counter()) -- NotImplemented not returned, TypeError not raised
ior(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised 
(should have worked)
isub(Counter()) -- Exception <'subtype' object has no attribute 'items'> raised 
(should have worked)


Two observations:

  - __ipow__ doesn't seem to behave properly in the 3.x line (that error 
doesn't show up when testing against 2.7)

  - Counter should be returning NotImplemented instead of raising an 
AttributeError, for three reasons [4]:
- a TypeError is more appropriate
- subclasses /cannot/ work with the current implementation
- __iand__ is currently a silent failure if the Counter is empty, and the 
other operand should trigger a failure

Back to the main point...

So, if my understanding is correct:

  - NotImplemented is used to signal Python that the requested operation could 
not be performed
  - it should be used by the binary special methods to signal type mismatch 
failure, so any subclass gets a chance to work.

Is my understanding correct?  Is this already in the docs somewhere, and I just 
missed it?

--
~Ethan~

[1] at least, it's the main reason in my code
[2] usually a TypeError, stating either that the operation is not supported, or 
the types are unorderable
[3] test script at the end
[4] https://bugs.python.org/issue22766 [returning NotImplemented was rejected]

-- 8< 

from collections import Counter, defaultdict, deque, OrderedDict
from fractions import Fraction
from decimal import Decimal
from enum import Enum
import operator
import sys

py_ver = sys.version_info[:2]

types = (
bytes, bytearray, str, dict, list, tuple,
Enum, Counter, defaultdict, deque, OrderedDict,
)
numeric_types = int, float, Decimal, Fraction

operators = (
'__add__', '__and__', '__floordiv__',
'__iadd__', '__iand__', '__ifloordiv__', '__ilshift__',
'__imod__', '__imul__', '__ior__', '__ipow__',
'__irshift__', '__isub__', '__itru

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread R. David Murray 
See issue 22766 for some background on this question.

On Mon, 03 Nov 2014 02:30:53 -0800, Ethan Furman  wrote:
> Just to be clear, this is about NotImplemented, not NotImplementedError.
> 
> tl;dr  When a binary operation fails, should an exception be raised or 
> NotImplemented returned?
> 
> 
> When a binary operation in Python is attempted, there are two possibilities:
> 
>- it can work
>- it can't work
> 
> The main reason [1] that it can't work is that the two operands are of 
> different types, and the first type does not know 
> how to deal with the second type.
> 
> The question then becomes: how does the first type tell Python that it cannot 
> perform the requested operation?  The most 
> obvious answer is to raise an exception, and TypeError is a good candidate.  
> The problem with the exception raising 
> approach is that once an exception is raised, Python doesn't try anything 
> else to make the operation work.
> 
> What's wrong with that?  Well, the second type might know how to perform the 
> operation, and in fact that is why we have 
> the reflected special methods, such as __radd__ and __rmod__ -- but if the 
> first type raises an exception the __rxxx__ 
> methods will not be tried.
> 
> Okay, how can the first type tell Python that it cannot do what is requested, 
> but to go ahead and check with the second 
> type to see if it does?  That is where NotImplemented comes in -- if a 
> special method (and only a special method) 
> returns NotImplemented then Python will check to see if there is anything 
> else it can do to make the operation succeed; 
> if all attempts return NotImplemented, then Python itself will raise an 
> appropriate exception [2].
> 
> In an effort to see how often NotImplemented is currently being returned I 
> crafted a test script [3] to test the types 
> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict, deque, 
> and OrderedDict with the operations for 
> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__, 
> __ilshift__, __imod__, __imul__, __ior__, __ipow__, 
> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__, __mul__, 
> __or__, __pow__, __rshift__, __sub__, 
> __truediv__, and __xor__.
> 
> Here are the results of the 275 tests:
> 
> testing control...
> 
> ipow -- Exception  'subtype'> raised
> errors in Control -- misunderstanding or bug?
> 
> testing types against a foreign class
> 
> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute 
> 'items'> raised instead of TypeError
> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute 
> 'items'> raised instead of TypeError
> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute 
> 'items'> raised instead of TypeError
> 
> 
> testing types against a subclass
> 
> mod(str()) -- NotImplemented not returned, TypeError not raised
> 
> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'> 
> raised (should have worked)
> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'> 
> raised (should have worked)
> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'> 
> raised (should have worked)
> 
> 
> Two observations:
> 
>- __ipow__ doesn't seem to behave properly in the 3.x line (that error 
> doesn't show up when testing against 2.7)
> 
>- Counter should be returning NotImplemented instead of raising an 
> AttributeError, for three reasons [4]:
>  - a TypeError is more appropriate
>  - subclasses /cannot/ work with the current implementation
>  - __iand__ is currently a silent failure if the Counter is empty, and 
> the other operand should trigger a failure
> 
> Back to the main point...
> 
> So, if my understanding is correct:
> 
>- NotImplemented is used to signal Python that the requested operation 
> could not be performed
>- it should be used by the binary special methods to signal type mismatch 
> failure, so any subclass gets a chance to work.
> 
> Is my understanding correct?  Is this already in the docs somewhere, and I 
> just missed it?
> 
> --
> ~Ethan~
> 
> [1] at least, it's the main reason in my code
> [2] usually a TypeError, stating either that the operation is not supported, 
> or the types are unorderable
> [3] test script at the end
> [4] https://bugs.python.org/issue22766 [returning NotImplemented was rejected]
> 
> -- 8< 
> 
> from collections import Counter, defaultdict, deque, OrderedDict
> from fractions import Fraction
> from decimal import Decimal
> from enum import Enum
> import operator
> import sys
> 
> py_ver = sys.ve

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Brett Cannon 
On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman  wrote:

> Just to be clear, this is about NotImplemented, not NotImplementedError.
>
> tl;dr  When a binary operation fails, should an exception be raised or
> NotImplemented returned?
>

The docs for NotImplemented suggest it's only for rich comparison methods
and not all binary operators:
https://docs.python.org/3/library/constants.html#NotImplemented . But then
had I not read that I would have said all binary operator methods should
return NotImplemented when the types are incompatible.

-Brett


>
>
> When a binary operation in Python is attempted, there are two
> possibilities:
>
>- it can work
>- it can't work
>
> The main reason [1] that it can't work is that the two operands are of
> different types, and the first type does not know
> how to deal with the second type.
>
> The question then becomes: how does the first type tell Python that it
> cannot perform the requested operation?  The most
> obvious answer is to raise an exception, and TypeError is a good
> candidate.  The problem with the exception raising
> approach is that once an exception is raised, Python doesn't try anything
> else to make the operation work.
>
> What's wrong with that?  Well, the second type might know how to perform
> the operation, and in fact that is why we have
> the reflected special methods, such as __radd__ and __rmod__ -- but if the
> first type raises an exception the __rxxx__
> methods will not be tried.
>
> Okay, how can the first type tell Python that it cannot do what is
> requested, but to go ahead and check with the second
> type to see if it does?  That is where NotImplemented comes in -- if a
> special method (and only a special method)
> returns NotImplemented then Python will check to see if there is anything
> else it can do to make the operation succeed;
> if all attempts return NotImplemented, then Python itself will raise an
> appropriate exception [2].
>
> In an effort to see how often NotImplemented is currently being returned I
> crafted a test script [3] to test the types
> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,
> deque, and OrderedDict with the operations for
> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,
> __ilshift__, __imod__, __imul__, __ior__, __ipow__,
> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,
> __mul__, __or__, __pow__, __rshift__, __sub__,
> __truediv__, and __xor__.
>
> Here are the results of the 275 tests:
> 
> 
> testing control...
>
> ipow -- Exception  and 'subtype'> raised
> errors in Control -- misunderstanding or bug?
>
> testing types against a foreign class
>
> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> 'items'> raised instead of TypeError
> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> 'items'> raised instead of TypeError
> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> 'items'> raised instead of TypeError
>
>
> testing types against a subclass
>
> mod(str()) -- NotImplemented not returned, TypeError not raised
>
> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> raised (should have worked)
> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> raised (should have worked)
> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> raised (should have worked)
> 
> 
>
> Two observations:
>
>- __ipow__ doesn't seem to behave properly in the 3.x line (that error
> doesn't show up when testing against 2.7)
>
>- Counter should be returning NotImplemented instead of raising an
> AttributeError, for three reasons [4]:
>  - a TypeError is more appropriate
>  - subclasses /cannot/ work with the current implementation
>  - __iand__ is currently a silent failure if the Counter is empty, and
> the other operand should trigger a failure
>
> Back to the main point...
>
> So, if my understanding is correct:
>
>- NotImplemented is used to signal Python that the requested operation
> could not be performed
>- it should be used by the binary special methods to signal type
> mismatch failure, so any subclass gets a chance to work.
>
> Is my understanding correct?  Is this already in the docs somewhere, and I
> just missed it?
>
> --
> ~Ethan~
>
> [1] at least, it's the main reason in my code
> [2] usually a TypeError, stating either that the operation is not
> supported, or the types are unorderable
> [3] test script at the end
> [4] https://bugs.python.org/issue22766 [returning NotImplemented was
> rejected]
>
> -- 8< 
>

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread R. David Murray 
On Mon, 03 Nov 2014 15:05:31 +, Brett Cannon  wrote:
> On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman  wrote:
> 
> > Just to be clear, this is about NotImplemented, not NotImplementedError.
> >
> > tl;dr  When a binary operation fails, should an exception be raised or
> > NotImplemented returned?
> >
> 
> The docs for NotImplemented suggest it's only for rich comparison methods
> and not all binary operators:
> https://docs.python.org/3/library/constants.html#NotImplemented . But then
> had I not read that I would have said all binary operator methods should
> return NotImplemented when the types are incompatible.

Ethan opened an issue and then changed those docs, but I now believe
that the docs should be changed back (see the discussion in issue
22766).

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Ethan Furman 

On 11/03/2014 08:12 AM, R. David Murray wrote:

On Mon, 03 Nov 2014 15:05:31 +, Brett Cannon  wrote:

On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman  wrote:


Just to be clear, this is about NotImplemented, not NotImplementedError.

tl;dr  When a binary operation fails, should an exception be raised or
NotImplemented returned?



The docs for NotImplemented suggest it's only for rich comparison methods
and not all binary operators:
https://docs.python.org/3/library/constants.html#NotImplemented . But then
had I not read that I would have said all binary operator methods should
return NotImplemented when the types are incompatible.


Ethan opened an issue and then changed those docs, but I now believe
that the docs should be changed back (see the discussion in issue
22766).


I was wondering myself, which is why I started this thread.

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Guido van Rossum 
Gotta be brief,  but NotImplemented is for all binary ops. Power may be an
exception because it's ternary?
On Nov 3, 2014 8:08 AM, "Brett Cannon"  wrote:

>
>
> On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman  wrote:
>
>> Just to be clear, this is about NotImplemented, not NotImplementedError.
>>
>> tl;dr  When a binary operation fails, should an exception be raised or
>> NotImplemented returned?
>>
>
> The docs for NotImplemented suggest it's only for rich comparison methods
> and not all binary operators:
> https://docs.python.org/3/library/constants.html#NotImplemented . But
> then had I not read that I would have said all binary operator methods
> should return NotImplemented when the types are incompatible.
>
> -Brett
>
>
>>
>>
>> When a binary operation in Python is attempted, there are two
>> possibilities:
>>
>>- it can work
>>- it can't work
>>
>> The main reason [1] that it can't work is that the two operands are of
>> different types, and the first type does not know
>> how to deal with the second type.
>>
>> The question then becomes: how does the first type tell Python that it
>> cannot perform the requested operation?  The most
>> obvious answer is to raise an exception, and TypeError is a good
>> candidate.  The problem with the exception raising
>> approach is that once an exception is raised, Python doesn't try anything
>> else to make the operation work.
>>
>> What's wrong with that?  Well, the second type might know how to perform
>> the operation, and in fact that is why we have
>> the reflected special methods, such as __radd__ and __rmod__ -- but if
>> the first type raises an exception the __rxxx__
>> methods will not be tried.
>>
>> Okay, how can the first type tell Python that it cannot do what is
>> requested, but to go ahead and check with the second
>> type to see if it does?  That is where NotImplemented comes in -- if a
>> special method (and only a special method)
>> returns NotImplemented then Python will check to see if there is anything
>> else it can do to make the operation succeed;
>> if all attempts return NotImplemented, then Python itself will raise an
>> appropriate exception [2].
>>
>> In an effort to see how often NotImplemented is currently being returned
>> I crafted a test script [3] to test the types
>> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,
>> deque, and OrderedDict with the operations for
>> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,
>> __ilshift__, __imod__, __imul__, __ior__, __ipow__,
>> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,
>> __mul__, __or__, __pow__, __rshift__, __sub__,
>> __truediv__, and __xor__.
>>
>> Here are the results of the 275 tests:
>> 
>> 
>> testing control...
>>
>> ipow -- Exception > and 'subtype'> raised
>> errors in Control -- misunderstanding or bug?
>>
>> testing types against a foreign class
>>
>> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute
>> 'items'> raised instead of TypeError
>> iand(Counter()) -- NotImplemented not returned, TypeError not raised
>> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute
>> 'items'> raised instead of TypeError
>> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute
>> 'items'> raised instead of TypeError
>>
>>
>> testing types against a subclass
>>
>> mod(str()) -- NotImplemented not returned, TypeError not raised
>>
>> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'>
>> raised (should have worked)
>> iand(Counter()) -- NotImplemented not returned, TypeError not raised
>> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'>
>> raised (should have worked)
>> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'>
>> raised (should have worked)
>> 
>> 
>>
>> Two observations:
>>
>>- __ipow__ doesn't seem to behave properly in the 3.x line (that error
>> doesn't show up when testing against 2.7)
>>
>>- Counter should be returning NotImplemented instead of raising an
>> AttributeError, for three reasons [4]:
>>  - a TypeError is more appropriate
>>  - subclasses /cannot/ work with the current implementation
>>  - __iand__ is currently a silent failure if the Counter is empty,
>> and the other operand should trigger a failure
>>
>> Back to the main point...
>>
>> So, if my understanding is correct:
>>
>>- NotImplemented is used to signal Python that the requested operation
>> could not be performed
>>- it should be used by the binary special methods to signal type
>> mismatch failure, so any subclass gets a chance to work.
>>
>> Is my understanding correct?  Is this already in the docs somewhere, and
>> I just missed it?
>>
>> --
>> ~Ethan~
>>
>> [1] at least, it's the main reason in my code
>> [2] usually a T

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Antoine Pitrou 
On Mon, 3 Nov 2014 08:48:07 -0800
Guido van Rossum  wrote:
> Gotta be brief,  but NotImplemented is for all binary ops.

Even in-place ops?

Regards

Antoine.


> Power may be an
> exception because it's ternary?
> On Nov 3, 2014 8:08 AM, "Brett Cannon"  wrote:
> 
> >
> >
> > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman  wrote:
> >
> >> Just to be clear, this is about NotImplemented, not NotImplementedError.
> >>
> >> tl;dr  When a binary operation fails, should an exception be raised or
> >> NotImplemented returned?
> >>
> >
> > The docs for NotImplemented suggest it's only for rich comparison methods
> > and not all binary operators:
> > https://docs.python.org/3/library/constants.html#NotImplemented . But
> > then had I not read that I would have said all binary operator methods
> > should return NotImplemented when the types are incompatible.
> >
> > -Brett
> >
> >
> >>
> >>
> >> When a binary operation in Python is attempted, there are two
> >> possibilities:
> >>
> >>- it can work
> >>- it can't work
> >>
> >> The main reason [1] that it can't work is that the two operands are of
> >> different types, and the first type does not know
> >> how to deal with the second type.
> >>
> >> The question then becomes: how does the first type tell Python that it
> >> cannot perform the requested operation?  The most
> >> obvious answer is to raise an exception, and TypeError is a good
> >> candidate.  The problem with the exception raising
> >> approach is that once an exception is raised, Python doesn't try anything
> >> else to make the operation work.
> >>
> >> What's wrong with that?  Well, the second type might know how to perform
> >> the operation, and in fact that is why we have
> >> the reflected special methods, such as __radd__ and __rmod__ -- but if
> >> the first type raises an exception the __rxxx__
> >> methods will not be tried.
> >>
> >> Okay, how can the first type tell Python that it cannot do what is
> >> requested, but to go ahead and check with the second
> >> type to see if it does?  That is where NotImplemented comes in -- if a
> >> special method (and only a special method)
> >> returns NotImplemented then Python will check to see if there is anything
> >> else it can do to make the operation succeed;
> >> if all attempts return NotImplemented, then Python itself will raise an
> >> appropriate exception [2].
> >>
> >> In an effort to see how often NotImplemented is currently being returned
> >> I crafted a test script [3] to test the types
> >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,
> >> deque, and OrderedDict with the operations for
> >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,
> >> __ilshift__, __imod__, __imul__, __ior__, __ipow__,
> >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,
> >> __mul__, __or__, __pow__, __rshift__, __sub__,
> >> __truediv__, and __xor__.
> >>
> >> Here are the results of the 275 tests:
> >> 
> >> 
> >> testing control...
> >>
> >> ipow -- Exception  >> and 'subtype'> raised
> >> errors in Control -- misunderstanding or bug?
> >>
> >> testing types against a foreign class
> >>
> >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> >> 'items'> raised instead of TypeError
> >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> >> 'items'> raised instead of TypeError
> >> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> >> 'items'> raised instead of TypeError
> >>
> >>
> >> testing types against a subclass
> >>
> >> mod(str()) -- NotImplemented not returned, TypeError not raised
> >>
> >> iadd(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> >> raised (should have worked)
> >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> >> ior(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> >> raised (should have worked)
> >> isub(Counter()) -- Exception <'subtype' object has no attribute 'items'>
> >> raised (should have worked)
> >> 
> >> 
> >>
> >> Two observations:
> >>
> >>- __ipow__ doesn't seem to behave properly in the 3.x line (that error
> >> doesn't show up when testing against 2.7)
> >>
> >>- Counter should be returning NotImplemented instead of raising an
> >> AttributeError, for three reasons [4]:
> >>  - a TypeError is more appropriate
> >>  - subclasses /cannot/ work with the current implementation
> >>  - __iand__ is currently a silent failure if the Counter is empty,
> >> and the other operand should trigger a failure
> >>
> >> Back to the main point...
> >>
> >> So, if my understanding is correct:
> >>
> >>- NotImplemented is used to signal Python that the requested operation
> >> coul

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Guido van Rossum 
Not those.
On Nov 3, 2014 8:56 AM, "Antoine Pitrou"  wrote:

> On Mon, 3 Nov 2014 08:48:07 -0800
> Guido van Rossum  wrote:
> > Gotta be brief,  but NotImplemented is for all binary ops.
>
> Even in-place ops?
>
> Regards
>
> Antoine.
>
>
> > Power may be an
> > exception because it's ternary?
> > On Nov 3, 2014 8:08 AM, "Brett Cannon"  wrote:
> >
> > >
> > >
> > > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman 
> wrote:
> > >
> > >> Just to be clear, this is about NotImplemented, not
> NotImplementedError.
> > >>
> > >> tl;dr  When a binary operation fails, should an exception be raised or
> > >> NotImplemented returned?
> > >>
> > >
> > > The docs for NotImplemented suggest it's only for rich comparison
> methods
> > > and not all binary operators:
> > > https://docs.python.org/3/library/constants.html#NotImplemented . But
> > > then had I not read that I would have said all binary operator methods
> > > should return NotImplemented when the types are incompatible.
> > >
> > > -Brett
> > >
> > >
> > >>
> > >>
> > >> When a binary operation in Python is attempted, there are two
> > >> possibilities:
> > >>
> > >>- it can work
> > >>- it can't work
> > >>
> > >> The main reason [1] that it can't work is that the two operands are of
> > >> different types, and the first type does not know
> > >> how to deal with the second type.
> > >>
> > >> The question then becomes: how does the first type tell Python that it
> > >> cannot perform the requested operation?  The most
> > >> obvious answer is to raise an exception, and TypeError is a good
> > >> candidate.  The problem with the exception raising
> > >> approach is that once an exception is raised, Python doesn't try
> anything
> > >> else to make the operation work.
> > >>
> > >> What's wrong with that?  Well, the second type might know how to
> perform
> > >> the operation, and in fact that is why we have
> > >> the reflected special methods, such as __radd__ and __rmod__ -- but if
> > >> the first type raises an exception the __rxxx__
> > >> methods will not be tried.
> > >>
> > >> Okay, how can the first type tell Python that it cannot do what is
> > >> requested, but to go ahead and check with the second
> > >> type to see if it does?  That is where NotImplemented comes in -- if a
> > >> special method (and only a special method)
> > >> returns NotImplemented then Python will check to see if there is
> anything
> > >> else it can do to make the operation succeed;
> > >> if all attempts return NotImplemented, then Python itself will raise
> an
> > >> appropriate exception [2].
> > >>
> > >> In an effort to see how often NotImplemented is currently being
> returned
> > >> I crafted a test script [3] to test the types
> > >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,
> > >> deque, and OrderedDict with the operations for
> > >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,
> > >> __ilshift__, __imod__, __imul__, __ior__, __ipow__,
> > >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,
> > >> __mul__, __or__, __pow__, __rshift__, __sub__,
> > >> __truediv__, and __xor__.
> > >>
> > >> Here are the results of the 275 tests:
> > >> 
> > >> 
> > >> testing control...
> > >>
> > >> ipow -- Exception  'Control'
> > >> and 'subtype'> raised
> > >> errors in Control -- misunderstanding or bug?
> > >>
> > >> testing types against a foreign class
> > >>
> > >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> > >> 'items'> raised instead of TypeError
> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> > >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> > >> 'items'> raised instead of TypeError
> > >> isub(Counter()) -- Exception <'SomeOtherClass' object has no attribute
> > >> 'items'> raised instead of TypeError
> > >>
> > >>
> > >> testing types against a subclass
> > >>
> > >> mod(str()) -- NotImplemented not returned, TypeError not raised
> > >>
> > >> iadd(Counter()) -- Exception <'subtype' object has no attribute
> 'items'>
> > >> raised (should have worked)
> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
> > >> ior(Counter()) -- Exception <'subtype' object has no attribute
> 'items'>
> > >> raised (should have worked)
> > >> isub(Counter()) -- Exception <'subtype' object has no attribute
> 'items'>
> > >> raised (should have worked)
> > >> 
> > >> 
> > >>
> > >> Two observations:
> > >>
> > >>- __ipow__ doesn't seem to behave properly in the 3.x line (that
> error
> > >> doesn't show up when testing against 2.7)
> > >>
> > >>- Counter should be returning NotImplemented instead of raising an
> > >> AttributeError, for three reasons [4]:
> > >>  - a TypeError is more appropriate
> > >>  - subclasses /cannot/ work wi

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Guido van Rossum 
Sorry, was too quick.  For immutable types __iop__ may not exist and then
the fallback machinery should work normally using NotImplemented. But if
__iop__ exists it can choose not to allow __rop__, because the type would
presumably change. This is probably more predictable. I don't even know if
the byte code interpreter looks for Not implemented from __iop__.
On Nov 3, 2014 9:00 AM, "Guido van Rossum"  wrote:

> Not those.
> On Nov 3, 2014 8:56 AM, "Antoine Pitrou"  wrote:
>
>> On Mon, 3 Nov 2014 08:48:07 -0800
>> Guido van Rossum  wrote:
>> > Gotta be brief,  but NotImplemented is for all binary ops.
>>
>> Even in-place ops?
>>
>> Regards
>>
>> Antoine.
>>
>>
>> > Power may be an
>> > exception because it's ternary?
>> > On Nov 3, 2014 8:08 AM, "Brett Cannon"  wrote:
>> >
>> > >
>> > >
>> > > On Mon Nov 03 2014 at 5:31:21 AM Ethan Furman 
>> wrote:
>> > >
>> > >> Just to be clear, this is about NotImplemented, not
>> NotImplementedError.
>> > >>
>> > >> tl;dr  When a binary operation fails, should an exception be raised
>> or
>> > >> NotImplemented returned?
>> > >>
>> > >
>> > > The docs for NotImplemented suggest it's only for rich comparison
>> methods
>> > > and not all binary operators:
>> > > https://docs.python.org/3/library/constants.html#NotImplemented . But
>> > > then had I not read that I would have said all binary operator methods
>> > > should return NotImplemented when the types are incompatible.
>> > >
>> > > -Brett
>> > >
>> > >
>> > >>
>> > >>
>> > >> When a binary operation in Python is attempted, there are two
>> > >> possibilities:
>> > >>
>> > >>- it can work
>> > >>- it can't work
>> > >>
>> > >> The main reason [1] that it can't work is that the two operands are
>> of
>> > >> different types, and the first type does not know
>> > >> how to deal with the second type.
>> > >>
>> > >> The question then becomes: how does the first type tell Python that
>> it
>> > >> cannot perform the requested operation?  The most
>> > >> obvious answer is to raise an exception, and TypeError is a good
>> > >> candidate.  The problem with the exception raising
>> > >> approach is that once an exception is raised, Python doesn't try
>> anything
>> > >> else to make the operation work.
>> > >>
>> > >> What's wrong with that?  Well, the second type might know how to
>> perform
>> > >> the operation, and in fact that is why we have
>> > >> the reflected special methods, such as __radd__ and __rmod__ -- but
>> if
>> > >> the first type raises an exception the __rxxx__
>> > >> methods will not be tried.
>> > >>
>> > >> Okay, how can the first type tell Python that it cannot do what is
>> > >> requested, but to go ahead and check with the second
>> > >> type to see if it does?  That is where NotImplemented comes in -- if
>> a
>> > >> special method (and only a special method)
>> > >> returns NotImplemented then Python will check to see if there is
>> anything
>> > >> else it can do to make the operation succeed;
>> > >> if all attempts return NotImplemented, then Python itself will raise
>> an
>> > >> appropriate exception [2].
>> > >>
>> > >> In an effort to see how often NotImplemented is currently being
>> returned
>> > >> I crafted a test script [3] to test the types
>> > >> bytes, bytearray, str, dict, list, tuple, Enum, Counter, defaultdict,
>> > >> deque, and OrderedDict with the operations for
>> > >> __add__, __and__, __floordiv__, __iadd__, __iand__, __ifloordiv__,
>> > >> __ilshift__, __imod__, __imul__, __ior__, __ipow__,
>> > >> __irshift__, __isub__, __itruediv__, __ixor__, __lshift__, __mod__,
>> > >> __mul__, __or__, __pow__, __rshift__, __sub__,
>> > >> __truediv__, and __xor__.
>> > >>
>> > >> Here are the results of the 275 tests:
>> > >> 
>> > >> 
>> > >> testing control...
>> > >>
>> > >> ipow -- Exception > 'Control'
>> > >> and 'subtype'> raised
>> > >> errors in Control -- misunderstanding or bug?
>> > >>
>> > >> testing types against a foreign class
>> > >>
>> > >> iadd(Counter()) -- Exception <'SomeOtherClass' object has no
>> attribute
>> > >> 'items'> raised instead of TypeError
>> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
>> > >> ior(Counter()) -- Exception <'SomeOtherClass' object has no attribute
>> > >> 'items'> raised instead of TypeError
>> > >> isub(Counter()) -- Exception <'SomeOtherClass' object has no
>> attribute
>> > >> 'items'> raised instead of TypeError
>> > >>
>> > >>
>> > >> testing types against a subclass
>> > >>
>> > >> mod(str()) -- NotImplemented not returned, TypeError not raised
>> > >>
>> > >> iadd(Counter()) -- Exception <'subtype' object has no attribute
>> 'items'>
>> > >> raised (should have worked)
>> > >> iand(Counter()) -- NotImplemented not returned, TypeError not raised
>> > >> ior(Counter()) -- Exception <'subtype' object has no attribute
>> 'items'>
>> > >> raised (should have worked)
>> > >> isub(Counter()) -- Exception <'subt

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Antoine Pitrou 
On Mon, 3 Nov 2014 09:05:43 -0800
Guido van Rossum  wrote:
> Sorry, was too quick.  For immutable types __iop__ may not exist and then
> the fallback machinery should work normally using NotImplemented. But if
> __iop__ exists it can choose not to allow __rop__, because the type would
> presumably change. This is probably more predictable. I don't even know if
> the byte code interpreter looks for Not implemented from __iop__.

Apparently it can tell it to fallback on __op__:

>>> class C(list):
...   def __iadd__(self, other):
... print("here")
... return NotImplemented
... 
>>> c = C()
>>> c += [1]
here
>>> c
[1]
>>> type(c)



Regards

Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Guido van Rossum 
That must be so that an immutable type can still implement __iop__ as an
optimization.

On Mon, Nov 3, 2014 at 9:10 AM, Antoine Pitrou  wrote:

> On Mon, 3 Nov 2014 09:05:43 -0800
> Guido van Rossum  wrote:
> > Sorry, was too quick.  For immutable types __iop__ may not exist and then
> > the fallback machinery should work normally using NotImplemented. But if
> > __iop__ exists it can choose not to allow __rop__, because the type would
> > presumably change. This is probably more predictable. I don't even know
> if
> > the byte code interpreter looks for Not implemented from __iop__.
>
> Apparently it can tell it to fallback on __op__:
>
> >>> class C(list):
> ...   def __iadd__(self, other):
> ... print("here")
> ... return NotImplemented
> ...
> >>> c = C()
> >>> c += [1]
> here
> >>> c
> [1]
> >>> type(c)
> 
>
>
> Regards
>
> Antoine.
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Ethan Furman 

Summary:

NotImplemented _should_ be used by the normal and reflected binary methods 
(__lt__, __add__, __xor__, __rsub__, etc.)

NotImplemented _may_ be used by the in-place binary methods (__iadd__, __ixor__, etc.), but the in-place methods are 
also free to raise an exception.


Correct?

--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The role of NotImplemented: What is it for and when should it be used?

2014-11-03 Thread Guido van Rossum 
Sounds good!

On Mon, Nov 3, 2014 at 11:33 AM, Ethan Furman  wrote:

> Summary:
>
> NotImplemented _should_ be used by the normal and reflected binary methods
> (__lt__, __add__, __xor__, __rsub__, etc.)
>
> NotImplemented _may_ be used by the in-place binary methods (__iadd__,
> __ixor__, etc.), but the in-place methods are also free to raise an
> exception.
>
> Correct?
>
> --
> ~Ethan~
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [OFF-TOPIC] It is true that is impossible write in binary code, the lowest level of programming that you can write is in hex code?

2014-11-03 Thread françai s 
I intend to write in lowest level of computer programming as a hobby.

It is true that is impossible write in binary code, the lowest level of
programming that you can write is in hex code?

What is the lowest level of programming computers that you can write ?

Is binary code?

Is hex code?

Is another machine code? Honestly do not know if it is true that there is
another machine code beyond the binary and hex code.

Is Assembly?
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [OFF-TOPIC] It is true that is impossible write in binary code, the lowest level of programming that you can write is in hex code?

2014-11-03 Thread Ryan Gonzalez 
Wait...you sent this to 9fans AND Python Dev??? Why not ask on StackExchange
Programmers  or something?

On Mon, Nov 3, 2014 at 5:19 PM, françai s  wrote:

> I intend to write in lowest level of computer programming as a hobby.
>
> It is true that is impossible write in binary code, the lowest level of
> programming that you can write is in hex code?
>
> What is the lowest level of programming computers that you can write ?
>
> Is binary code?
>
> Is hex code?
>
> Is another machine code? Honestly do not know if it is true that there is
> another machine code beyond the binary and hex code.
>
> Is Assembly?
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>
>


-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [OFF-TOPIC] It is true that is impossible write in binary code, the lowest level of programming that you can write is in hex code?

2014-11-03 Thread Steven D'Aprano 
This is off-topic for this mailing list, as you know. There are some 
mailing lists which approve of off-topic conversations, but this is not 
one of those.

You could ask on the [email protected] mailing list, where it will 
still be off-topic, but the people there are more likely to answer. But 
even better would be to look for a mailing list or forum for assembly 
programming, machine code, or micro-code.


On Mon, Nov 03, 2014 at 09:19:46PM -0200, françai s wrote:
> I intend to write in lowest level of computer programming as a hobby.
[...]


-- 
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com