Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
2013/7/19 Ethan Furman :
> While working on issue #18508 I stumbled across this:
>
> Traceback (most recent call last):
> ...
> File "/usr/local/lib/python3.4/enum.py", line 417, in __new__
> if value in cls._value2member_map:
> TypeError: unhashable type: 'list'
>
> I'll wrap it in a try-except block, but I must admit I was surprised the
> answer wasn't False. After all, if the input is unhashable then obviously
> it's not in the dict; furthermore, if I were to compare the number 5 with a
> set() I would get False, not a TypeMismatch error, and dict keys are
> basically done by equality, the hash is just (?) a speed-up.
I'm not exactly sure what the last part of that sentence means.
Anyway, it's well established that operations on a key in a dict are
going to involve looking up the key, and thus hashing it. You wouldn't
expect, {}.get(unhashable, None) not to raise, right?
--
Regards,
Benjamin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
On 20 July 2013 09:47, Ethan Furman wrote:
> While working on issue #18508 I stumbled across this:
>
> Traceback (most recent call last):
> ...
> File "/usr/local/lib/python3.4/enum.py", line 417, in __new__
> if value in cls._value2member_map:
> TypeError: unhashable type: 'list'
>
> I'll wrap it in a try-except block, but I must admit I was surprised the
> answer wasn't False. After all, if the input is unhashable then obviously
> it's not in the dict; furthermore, if I were to compare the number 5 with a
> set() I would get False, not a TypeMismatch error, and dict keys are
> basically done by equality, the hash is just (?) a speed-up.
"in" is controlled entirely by the container. Both set() and dict()
consider passing them an unhashable object to be a categorical type
error:
>>> [] in set()
Traceback (most recent call last):
File "", line 1, in
TypeError: unhashable type: 'list'
>>> [] in {}
Traceback (most recent call last):
File "", line 1, in
TypeError: unhashable type: 'list'
Equality is different, because people do equality comparisons between
unlike types all the time, so we try *very* hard to ensure "I do not
understand this type" is reported as False rather than "I don't know"
(as indicated by an exception). It's basically the only place where we
do that - pretty much everywhere else, we err on the side of "I do not
understand what this even means" translating to an exception (Python 3
took another big step in that direction by making sure ordered
comparisons between unlike types threw TypeError by default, instead
of returning a dubious answer).
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
On 20 Jul, 2013, at 1:47, Ethan Furman wrote:
> While working on issue #18508 I stumbled across this:
>
> Traceback (most recent call last):
> ...
> File "/usr/local/lib/python3.4/enum.py", line 417, in __new__
>if value in cls._value2member_map:
> TypeError: unhashable type: 'list'
>
> I'll wrap it in a try-except block, but I must admit I was surprised the
> answer wasn't False. After all, if the input is unhashable then obviously
> it's not in the dict; furthermore, if I were to compare the number 5 with a
> set() I would get False, not a TypeMismatch error, and dict keys are
> basically done by equality, the hash is just (?) a speed-up.
Not quite, there are some objects that compare equal without both of them being
hashable:
>>> frozenset([1,2]) == set([1,2])
True
>>> dct = { frozenset([1,2]): 1 }
>>> frozenset([1,2]) in dct
True
>>> set([1,2]) in dct
Traceback (most recent call last):
File "", line 1, in
TypeError: unhashable type: 'set'
It would be strange if the last test would return False instead of raising an
error.
Ronald
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
On Fri, 19 Jul 2013 16:47:36 -0700 Ethan Furman wrote: > While working on issue #18508 I stumbled across this: > > Traceback (most recent call last): > ... >File "/usr/local/lib/python3.4/enum.py", line 417, in __new__ > if value in cls._value2member_map: > TypeError: unhashable type: 'list' > > I'll wrap it in a try-except block, but I must admit I was surprised the > answer wasn't False. After all, if the input > is unhashable then obviously it's not in the dict; furthermore, if I were to > compare the number 5 with a set() I would > get False, not a TypeMismatch error, and dict keys are basically done by > equality, the hash is just (?) a speed-up. Testing containment of an unhashable key in a dict is a programming error, so I'd prefer that we keep the TypeError here. Same reasoning as for: >>> 5 in "abc" Traceback (most recent call last): File "", line 1, in TypeError: 'in ' requires string as left operand, not int Sure, 5 technically isn't in "abc" and therefore the "in" operator could return False, but it's better to highlight the programming error. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dash
19.07.13 22:32, Ben Finney написав(ла): Serhiy Storchaka writes: I'm asking only about this case, when the dash is used to denote a break in a sentence or to set off parenthetical statements. That's two separate cases: * denote a break in a sentence * set off parenthetical statements In Wikipedia they considered as one case. In my experience, an em dash is commonly used for the former, and en dashes commonly used for the latter. Using the same dash for both is unusual (and IMO needlessly ambiguous to the reader). This is definitely should be described in the official guide. I never heard about this. Currently Python documentation uses fourth variants: All your examples are only for parenthetical statements. Can you find examples of the former, where a break (not parenthetical) in the sentence is intended? (2) and (4) are from Wikipedia (which consider it as one case). Here are real random examples from Python documentation: 1. "Common installation tools such as ``Distribute`` and ``pip`` work as expected with venvs - i.e. when a venv is active, they install Python packages into the venv without needing to be told to do so explicitly." 2. "Enter *string* in the table of "interned" strings and return the interned string which is *string* itself or a copy." 3. "The :class:`set` type is mutable --- the contents can be changed using methods like :meth:`add` and :meth:`remove`." 4. "Three locking mechanisms are used---dot locking and, if available, the :c:func:`flock` and :c:func:`lockf` system calls." ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
20.07.13 15:12, christian.heimes написав(ла): http://hg.python.org/cpython/rev/c92f4172d122 changeset: 84723:c92f4172d122 user:Christian Heimes date:Sat Jul 20 14:11:28 2013 +0200 summary: Use strncat() instead of strcat() to silence some warnings. CID 486616, CID 486617, CID 486615 files: Modules/ossaudiodev.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -245,7 +245,7 @@ int arg; assert(strlen(fname) <= 30); -strcat(argfmt, fname); +strncat(argfmt, fname, 30); if (!PyArg_ParseTuple(args, argfmt, &arg)) return NULL; @@ -270,7 +270,7 @@ int arg = 0; assert(strlen(fname) <= 30); -strcat(argfmt, fname); +strncat(argfmt, fname, 30); if (!PyArg_ParseTuple(args, argfmt, &arg)) return NULL; @@ -290,7 +290,7 @@ int rv; assert(strlen(fname) <= 30); -strcat(argfmt, fname); +strncat(argfmt, fname, 30); if (!PyArg_ParseTuple(args, argfmt)) return NULL; This will wrong when strlen(fname) is 30. strncat() will copy only 30 bytes, without terminal NUL. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
On Sat, 20 Jul 2013 15:23:46 +0300 Serhiy Storchaka wrote: > 20.07.13 15:12, christian.heimes написав(ла): > > http://hg.python.org/cpython/rev/c92f4172d122 > > changeset: 84723:c92f4172d122 > > user:Christian Heimes > > date:Sat Jul 20 14:11:28 2013 +0200 > > summary: > >Use strncat() instead of strcat() to silence some warnings. > > CID 486616, CID 486617, CID 486615 > > [...] > > This will wrong when strlen(fname) is 30. strncat() will copy only 30 > bytes, without terminal NUL. So, for the record, this is roughly how Rasmus Lerdorf introduced a security hole in PHP 5.3.7: "For people asking me out-of-band what the screw-up was, it was pretty simple. I changed this code: memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); strcat(passwd, "$"); to: memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); strlcat(passwd, "$", 1); **because the Coverity static analyzer warned about using strcat** [emphasis mine] and we generally try to avoid naked strcat/strcpy in the codebase even though in this case it is safe to do." https://plus.google.com/113641248237520845183/posts/g68d9RvRA1i Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
20.07.13 15:36, Antoine Pitrou написав(ла): On Sat, 20 Jul 2013 15:23:46 +0300 Serhiy Storchaka wrote: 20.07.13 15:12, christian.heimes написав(ла): http://hg.python.org/cpython/rev/c92f4172d122 changeset: 84723:c92f4172d122 user:Christian Heimes date:Sat Jul 20 14:11:28 2013 +0200 summary: Use strncat() instead of strcat() to silence some warnings. CID 486616, CID 486617, CID 486615 [...] This will wrong when strlen(fname) is 30. strncat() will copy only 30 bytes, without terminal NUL. So, for the record, this is roughly how Rasmus Lerdorf introduced a security hole in PHP 5.3.7: "For people asking me out-of-band what the screw-up was, it was pretty simple. I changed this code: memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); strcat(passwd, "$"); to: memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); strlcat(passwd, "$", 1); **because the Coverity static analyzer warned about using strcat** [emphasis mine] and we generally try to avoid naked strcat/strcpy in the codebase even though in this case it is safe to do." https://plus.google.com/113641248237520845183/posts/g68d9RvRA1i strlcat != strncat. strlcat(dst, src, 1) actually do nothing. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
On Sat, 20 Jul 2013 15:48:09 +0300 Serhiy Storchaka wrote: > 20.07.13 15:36, Antoine Pitrou написав(ла): > > On Sat, 20 Jul 2013 15:23:46 +0300 > > Serhiy Storchaka wrote: > >> 20.07.13 15:12, christian.heimes написав(ла): > >>> http://hg.python.org/cpython/rev/c92f4172d122 > >>> changeset: 84723:c92f4172d122 > >>> user:Christian Heimes > >>> date:Sat Jul 20 14:11:28 2013 +0200 > >>> summary: > >>> Use strncat() instead of strcat() to silence some warnings. > >>> CID 486616, CID 486617, CID 486615 > >>> > > [...] > >> > >> This will wrong when strlen(fname) is 30. strncat() will copy only 30 > >> bytes, without terminal NUL. > > > > So, for the record, this is roughly how Rasmus Lerdorf introduced a > > security hole in PHP 5.3.7: > > > > > > "For people asking me out-of-band what the screw-up was, it was pretty > > simple. I changed this code: > > > > memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); > > strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); > > strcat(passwd, "$"); > > > > to: > > > > memcpy(passwd, MD5_MAGIC, MD5_MAGIC_LEN); > > strlcpy(passwd + MD5_MAGIC_LEN, sp, sl + 1); > > strlcat(passwd, "$", 1); > > > > **because the Coverity static analyzer warned about using strcat** > > [emphasis mine] and we generally try to avoid naked strcat/strcpy in the > > codebase even though in this case it is safe to do." > > > > https://plus.google.com/113641248237520845183/posts/g68d9RvRA1i > > strlcat != strncat. strlcat(dst, src, 1) actually do nothing. This is true. But the trigger in the "fix" was the same (trying to suppress a Coverity warning about strcat). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
> This will wrong when strlen(fname) is 30. strncat() will copy only 30 > bytes, without terminal NUL. That's not how strncat() works. strncat(dest, src, n) writes n+1 chars to the end of dest: n chars from src and +1 for the final NUL char. For this reason dest must be large enough to hold strlen(dest) + n + 1 chars. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
Am 20.07.2013 14:23, schrieb Serhiy Storchaka: > This will wrong when strlen(fname) is 30. strncat() will copy only 30 > bytes, without terminal NUL. http://linux.die.net/man/3/strncat The strncat() function is similar, except that * it will use at most n bytes from src; and * src does not need to be null-terminated if it contains n or more bytes. If src contains n or more bytes, strncat() writes n+1 bytes to dest (n from src plus the terminating null byte). Therefore, the size of dest must be at least strlen(dest)+n+1. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
20.07.13 16:27, Christian Heimes написав(ла): This will wrong when strlen(fname) is 30. strncat() will copy only 30 bytes, without terminal NUL. That's not how strncat() works. strncat(dest, src, n) writes n+1 chars to the end of dest: n chars from src and +1 for the final NUL char. For this reason dest must be large enough to hold strlen(dest) + n + 1 chars. Oh, true. strncat() always results NUL-terminated string. It's strncpy() can produce not NUL-terminated string. Sorry for noise. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.
Am 20.07.2013 15:56, schrieb Serhiy Storchaka: > Oh, true. strncat() always results NUL-terminated string. It's strncpy() > can produce not NUL-terminated string. Sorry for noise. You're welcome! Better safe than sorry. :) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
On 07/20/2013 03:21 AM, Ronald Oussoren wrote:
On 20 Jul, 2013, at 1:47, Ethan Furman wrote:
While working on issue #18508 I stumbled across this:
Traceback (most recent call last):
...
File "/usr/local/lib/python3.4/enum.py", line 417, in __new__
if value in cls._value2member_map:
TypeError: unhashable type: 'list'
I'll wrap it in a try-except block, but I must admit I was surprised the answer
wasn't False. After all, if the input is unhashable then obviously it's not in
the dict; furthermore, if I were to compare the number 5 with a set() I would
get False, not a TypeMismatch error, and dict keys are basically done by
equality, the hash is just (?) a speed-up.
Not quite, there are some objects that compare equal without both of them being
hashable:
frozenset([1,2]) == set([1,2])
True
dct = { frozenset([1,2]): 1 }
frozenset([1,2]) in dct
True
set([1,2]) in dct
Traceback (most recent call last):
File "", line 1, in
TypeError: unhashable type: 'set'
It would be strange if the last test would return False instead of raising an
error.
Yeah, that would be unfortunate.
Okay, I can see dict.__contains__ and friends raising a TypeError, but
what about dict.keys() ? In 2.x it returned a list, in 3.x it returns a
dict_keys, and dict_keys also raises an error instead of returning False.
--
~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dict __contains__ raises TypeError on unhashable input
On Sat, Jul 20, 2013 at 6:37 PM, Ethan Furman wrote:
> On 07/20/2013 03:21 AM, Ronald Oussoren wrote:
>>
>>
>> On 20 Jul, 2013, at 1:47, Ethan Furman wrote:
>>
>>> While working on issue #18508 I stumbled across this:
>>>
>>> Traceback (most recent call last):
>>> ...
>>> File "/usr/local/lib/python3.4/enum.py", line 417, in __new__
>>> if value in cls._value2member_map:
>>> TypeError: unhashable type: 'list'
>>>
>>> I'll wrap it in a try-except block, but I must admit I was surprised the
>>> answer wasn't False. After all, if the input is unhashable then obviously
>>> it's not in the dict; furthermore, if I were to compare the number 5 with a
>>> set() I would get False, not a TypeMismatch error, and dict keys are
>>> basically done by equality, the hash is just (?) a speed-up.
>>
>>
>> Not quite, there are some objects that compare equal without both of them
>> being hashable:
>>
> frozenset([1,2]) == set([1,2])
>>
>> True
>
> dct = { frozenset([1,2]): 1 }
> frozenset([1,2]) in dct
>>
>> True
>
> set([1,2]) in dct
>>
>> Traceback (most recent call last):
>>File "", line 1, in
>> TypeError: unhashable type: 'set'
>>
>> It would be strange if the last test would return False instead of raising
>> an error.
>
>
> Yeah, that would be unfortunate.
>
> Okay, I can see dict.__contains__ and friends raising a TypeError, but what
> about dict.keys() ? In 2.x it returned a list, in 3.x it returns a
> dict_keys, and dict_keys also raises an error instead of returning False.
The same reasoning applies. And the Python 2 and 3 keys() methods are
so different that the backwards compatibility argument doesn't really
have any weight.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
