[Python-Dev] Real-world use of Counter

[Ethan Furman]

I'm looking for real-world uses of collections.Counter, specifically to see if anyone has been surprised by, or had to 
spend extra-time debugging, issues with the in-place operators.


If sufficient and/or compelling use-cases are uncovered, the behavior of 
Counter may be slightly modified.

Background:

Most Python data types will cause a TypeError to be raised if unusable types 
are passed in:

--> {'a': 0}.update(5)
TypeError: 'int' object is not iterable

--> [1, 2, 3].extend(3.14)
TypeError: 'float' object is not iterable

--> from collections import Counter
--> Counter() + [1, 2, 3]
TypeError: unsupported operand type(s) for +: 'Counter' and 'list'

Most Counter in-place methods also behave this way:

--> c /= [1, 2, 3]
TypeError: unsupported operand type(s) for /=: 'Counter' and 'list'

However, in the case of a handful of Counter in-place methods (+=, -=, &=, and 
|=), this is what happens instead:

--> c += [1, 2, 3]
AttributeError: 'list' object has no attribute 'items'

Even worse (in my opinion) is the case of an empty Counter `and`ed with an 
incompatible type:

--> c &= [1, 2, 3]
-->

No error is raised at all.

In order to avoid unnecessary code churn (the fix itself is quite simple), the maintainer of the collections module 
wants to know if anybody has actually been affected by these inconsistencies, and if so, whether it was a minor 
inconvenience, or a compelling use-case.


So, if this has bitten you, now is the time to speak up!  :)

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[MRAB]

On 2014-11-05 16:33, Ethan Furman wrote:

I'm looking for real-world uses of collections.Counter, specifically to see if 
anyone has been surprised by, or had to
spend extra-time debugging, issues with the in-place operators.

If sufficient and/or compelling use-cases are uncovered, the behavior of 
Counter may be slightly modified.

Background:

Most Python data types will cause a TypeError to be raised if unusable types 
are passed in:

--> {'a': 0}.update(5)
TypeError: 'int' object is not iterable

--> [1, 2, 3].extend(3.14)
TypeError: 'float' object is not iterable

--> from collections import Counter
--> Counter() + [1, 2, 3]
TypeError: unsupported operand type(s) for +: 'Counter' and 'list'

Most Counter in-place methods also behave this way:

--> c /= [1, 2, 3]
TypeError: unsupported operand type(s) for /=: 'Counter' and 'list'

However, in the case of a handful of Counter in-place methods (+=, -=, &=, and 
|=), this is what happens instead:

--> c += [1, 2, 3]
AttributeError: 'list' object has no attribute 'items'

Even worse (in my opinion) is the case of an empty Counter `and`ed with an 
incompatible type:

--> c &= [1, 2, 3]
-->

No error is raised at all.

In order to avoid unnecessary code churn (the fix itself is quite simple), the 
maintainer of the collections module
wants to know if anybody has actually been affected by these inconsistencies, 
and if so, whether it was a minor
inconvenience, or a compelling use-case.

So, if this has bitten you, now is the time to speak up!  :)


Whenever I've used the Counter class, I've known it was a counter, and
not tried to do any of the above, but at least, if I did, I'd get an
exception (or hope I do).

The final example, however, is odd. I think that one should be fixed.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[Raymond Hettinger]

> On Nov 5, 2014, at 8:33 AM, Ethan Furman  wrote:
> 
> I'm looking for real-world uses of collections.Counter, specifically to see 
> if anyone has been surprised by, or had to spend extra-time debugging, issues 
> with the in-place operators.

Please stop using the mailing lists as way to make an end-run around 
discussions on the tracker.  http://bugs.python.org/issue22766 
   

Also, as asked the question is a bit loaded.  Effectively, it asks "has anyone 
ever been surprised by an exception raised by a duck-typed function or method"?

The in-place operations on counters are duck-typed.  They are intended (by 
design) to work with ANY type that has an items() method.   The exception 
raised if doesn't have on is an AttributeError saying that the operand needs to 
have an items() method.

I do not want to change API for already deployed code just because you would 
rather see a TypeError instead.  Minor API changes (switching exception types) 
creates unnecessary consternation for users.

Please let this one die.  It seems to have become your pet project even after 
I've made a decision and explained my rationale.


Raymond___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[Ethan Furman]

On 11/05/2014 10:09 AM, MRAB wrote:

On 2014-11-05 16:33, Ethan Furman wrote:


Even worse (in my opinion) is the case of an empty Counter `and`ed with an 
incompatible type:

--> c &= [1, 2, 3]
-->

No error is raised at all.


The final example, however, is odd. I think that one should be fixed.


https://bugs.python.org/issue22801

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[Ethan Furman]

On 11/05/2014 10:56 AM, Raymond Hettinger wrote:


Please stop using the mailing lists as way to make an end-run around 
discussions on the tracker.
http://bugs.python.org/issue22766


You said that without compelling, real-world use cases you don't like to make 
changes.

The tracker has a very limited audience, while the mailing lists have a much greater chance of reaching the users who 
may actually have the use-cases you would consider.


You call it an end-run, I call it research.



Also, as asked the question is a bit loaded.  Effectively, it asks "has anyone 
ever been surprised by an exception
raised by a duck-typed function or method"?


Actually, it's asking, "Most other duck-typed methods will still raise a TypeError, but these few don't.  Has that ever 
been a problem for you?"




The in-place operations on counters are duck-typed.  They are intended (by 
design) to work with ANY type that has an
items() method.   The exception raised if doesn't have on is an AttributeError 
saying that the operand needs to have an
items() method.


It would still be duck-typed with a `hasattr` call on the second operand checking for the necessary method, a TypeError 
could just as easily state the problem is a missing `items()` method, and then those three [*] in-place methods would 
raise the same error as the 20(?) other Counter methods under similar conditions.




Please let this one die.  It seems to have become your pet project even after 
I've made a decision and explained my
rationale.


You indicated you might make a change with sufficient real-world use-cases, so I'm trying to find some.  If none show 
up, I'll let the matter drop.


--
~Ethan~

[*] Amusingly enough, there are four overridden in-place methods:  three raise an AttributeError, but one (&=) still 
raises a TypeError.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[R. David Murray]

On Wed, 05 Nov 2014 11:13:37 -0800, Ethan Furman  wrote:
> On 11/05/2014 10:56 AM, Raymond Hettinger wrote:
> > The in-place operations on counters are duck-typed.  They are intended (by 
> > design) to work with ANY type that has an
> > items() method.   The exception raised if doesn't have on is an 
> > AttributeError saying that the operand needs to have an
> > items() method.
> 
> It would still be duck-typed with a `hasattr` call on the second operand 
> checking for the necessary method, a TypeError 
> could just as easily state the problem is a missing `items()` method, and 
> then those three [*] in-place methods would 
> raise the same error as the 20(?) other Counter methods under similar 
> conditions.

The technique of duck typing as used in Python is you just let errors
bubble up when the input doesn't work.  Adding special case checks
should in general only be done if the resulting error message is
otherwise really confusing, which is not the case here.

As I said on the issue, there is no reason I can see to add extra code
just to turn an AttributeError into a TypeError.  The AttributeError
works just fine in letting you know your input type didn't work.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[Greg Ewing]

Ethan Furman wrote:
Actually, it's asking, "Most other duck-typed methods will still raise a 
TypeError, but these few don't.  Has that ever been a problem for you?"


I don't think I've *ever* been bothered by getting an
AttributeError instead of a TypeError or vice versa.
Both indicate bugs in my code, and I debug it by looking
at the code and traceback; I don't try to guess the problem
based solely on the exception type.

In this case the code would have to go out of its way
to turn an AttributeError into a TypeError. I don't
think the cost of that is worth whatever small benefit
there might be, if any.

Summary: Looks fine to me as it is.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[Alexander Belopolsky]

On Wed, Nov 5, 2014 at 2:47 PM, R. David Murray 
wrote:

> As I said on the issue, there is no reason I can see to add extra code
> just to turn an AttributeError into a TypeError.  The AttributeError
> works just fine in letting you know your input type didn't work.
>

+1

Unlike ValueError or LookupError, TypeError and AttributeError indicate a
logical problem with the code rather than an issue with the user input.
>From the programmer perspective, any code that catches and mutates
exceptions is a nuisance.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com