[Python-Dev] bugs.python.org

[Mark Lawrence]

Suffering from dead parrot syndrome?  Kiss of life please :)

Kindest regards.

Mark Lawrence.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

Hello,

I know the issue has been discussed several times already, however I couldn't 
find any reasonable explanation of its strange behaviour.  The main problem 
with 'hasattr' function is that is swallows all exceptions derived from 
Exception class.  It's a good thing that it doesn't do that with BaseException 
as it was fixed not a long time ago, but it's definitely not enough.

First of all, this behaviour of 'hasattr' contradicts with the very core 
principle of python: "Errors should never pass silently."  And since 'hasattr' 
function is in builtins module and is a widely used function it impacts the 
whole language.

Secondly, take a look at the following:

>>> class Test:
... @property
... def attr(self):
... self['foo']
... 
>>> hasattr(Test(), 'attr')
False

There can be any exception instead of KeyError in the above snippet of code, 
but this small example shows how 'hasattr': misleadingly breaks the code logic 
(1) and masks bug (2).  And that's the simplest possible example, there are 
much more in real life.

While (1) is maybe acceptable for someone, there is no excuse for the (2).  
Moreover, current 'hasattr' behaviour tremendously complicates use of 
'__getattribute__' magic.  And forget about importlib magic with LazyImports, 
one 'hasattr' ruins everything by catching ImportError.


To conclude:

1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
only AttributeError exceptions (exactly like 'getattr').  Probably, Python 3.2 
release is our last chance.

2) If you afraid that this new behaviour will break too much python 2 code 
converted with 2to3, we can introduce another 'hasattr' function defined in 
2to3 module itself, and make it imported automatically in all files passed 
through 2to3 transformation pipeline.  This new function will mimic 'hasattr' 
behaviour from python 2 and converted code should work as expected.


-
Yury Selivanov,
Sprymix Inc.
+1-416-509-2807

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Yury Selivanov :
> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
> 3.2 release is our last chance.

I would be in support of that.

>
> 2) If you afraid that this new behaviour will break too much python 2 code 
> converted with 2to3, we can introduce another 'hasattr' function defined in 
> 2to3 module itself, and make it imported automatically in all files passed 
> through 2to3 transformation pipeline.  This new function will mimic 'hasattr' 
> behaviour from python 2 and converted code should work as expected.

But not this. Compatibility functions don't belong in 2to3.

-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

On 2010-08-23, at 10:46 AM, Benjamin Peterson wrote:

> 2010/8/23 Yury Selivanov :
>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
>> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
>> 3.2 release is our last chance.
> 
> I would be in support of that.
> 
>> 
>> 2) If you afraid that this new behaviour will break too much python 2 code 
>> converted with 2to3, we can introduce another 'hasattr' function defined in 
>> 2to3 module itself, and make it imported automatically in all files passed 
>> through 2to3 transformation pipeline.  This new function will mimic 
>> 'hasattr' behaviour from python 2 and converted code should work as expected.
> 
> But not this. Compatibility functions don't belong in 2to3.

There are many possible solutions for the Python 2 porting issue, this one was 
just one of them.  I was trying to make a point, that it's possible to somehow 
make porting process easier, and meanwhile fix Python 3.

-
Yury Selivanov
Sprymix Inc.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson  wrote:
> 2010/8/23 Yury Selivanov :
>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
>> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
>> 3.2 release is our last chance.
>
> I would be in support of that.

I am cautiously in favor. The existing behavior is definitely a
mistake and a trap. But it has been depended on for almost 20 years
now.

I recommend that you create a patch, apply it, run the *entire* stdlib
test suite and see how much breaks. That will give you an idea of the
damage to expect for 3rd party code.

>> 2) If you afraid that this new behaviour will break too much python 2 code 
>> converted with 2to3, we can introduce another 'hasattr' function defined in 
>> 2to3 module itself, and make it imported automatically in all files passed 
>> through 2to3 transformation pipeline.  This new function will mimic 
>> 'hasattr' behaviour from python 2 and converted code should work as expected.
>
> But not this. Compatibility functions don't belong in 2to3.

Indeed.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bugs.python.org

[R. David Murray]

On Mon, 23 Aug 2010 15:13:34 +0100, Mark Lawrence  
wrote:
> Suffering from dead parrot syndrome?  Kiss of life please :)

The hosting company has been notified.

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Guido van Rossum :
> On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson  
> wrote:
>> 2010/8/23 Yury Selivanov :
>>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to 
>>> swallow only AttributeError exceptions (exactly like 'getattr').  Probably, 
>>> Python 3.2 release is our last chance.
>>
>> I would be in support of that.
>
> I am cautiously in favor. The existing behavior is definitely a
> mistake and a trap. But it has been depended on for almost 20 years
> now.
>
> I recommend that you create a patch, apply it, run the *entire* stdlib
> test suite and see how much breaks. That will give you an idea of the
> damage to expect for 3rd party code.

The test suite passes complete without modification for me.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
> On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson  
> wrote:
>> 2010/8/23 Yury Selivanov :
>>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to 
>>> swallow only AttributeError exceptions (exactly like 'getattr').  Probably, 
>>> Python 3.2 release is our last chance.
>> I would be in support of that.
> 
> I am cautiously in favor. The existing behavior is definitely a
> mistake and a trap. But it has been depended on for almost 20 years
> now.
> 
> I recommend that you create a patch, apply it, run the *entire* stdlib
> test suite and see how much breaks. That will give you an idea of the
> damage to expect for 3rd party code.

Robust third-party code is written to avoid 'hasattr', for precisely
this reason.  Third-party code which relies on 'hasattr' to mask
non-AttributeErrors is broken alreay (it just doesn't show the borkedness).


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkxymr0ACgkQ+gerLs4ltQ4rQwCgyHJmqt2TefCgX2di5aJ92pVh
26YAnjKrBrK3gMs7ddo2wHtpT+iq2Mbg
=BFxu
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Terry Reedy]

On 8/23/2010 10:22 AM, Yury Selivanov wrote:


1) I propose to change 'hasattr' behaviour in Python 3, making it to
swallow only AttributeError exceptions (exactly like 'getattr').
Probably, Python 3.2 release is our last chance.


I gather that this amounts to changing "an exception" to 
"AttributeError" in "(This is implemented by calling getattr(object, 
name) and seeing whether it raises an exception or not.)" in both the 
doc and implementation, so that the implementation actually matches the 
claimed behavior "The result is True if the string is the name of one of 
the object’s attributes, False if not. " (and by reasonable implication, 
an exception if this cannot be determined).


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Raymond Hettinger]

On Aug 23, 2010, at 7:22 AM, Yury Selivanov wrote:

> I know the issue has been discussed several times already, however I couldn't 
> find any reasonable explanation of its strange behaviour.  The main problem 
> with 'hasattr' function is that is swallows all exceptions derived from 
> Exception class.  It's a good thing that it doesn't do that with 
> BaseException as it was fixed not a long time ago, but it's definitely not 
> enough.
> 
> First of all, this behaviour of 'hasattr' contradicts with the very core 
> principle of python: "Errors should never pass silently."  And since 
> 'hasattr' function is in builtins module and is a widely used function it 
> impacts the whole language.
> 
> Secondly, take a look at the following:
> 
 class Test:
>... @property
>... def attr(self):
>... self['foo']
>... 
 hasattr(Test(), 'attr')
>False
> 
> There can be any exception instead of KeyError in the above snippet of code, 
> but this small example shows how 'hasattr': misleadingly breaks the code 
> logic (1) and masks bug (2).  And that's the simplest possible example, there 
> are much more in real life.
> 
> While (1) is maybe acceptable for someone, there is no excuse for the (2).  
> Moreover, current 'hasattr' behaviour tremendously complicates use of 
> '__getattribute__' magic.  And forget about importlib magic with LazyImports, 
> one 'hasattr' ruins everything by catching ImportError.
> 
> 
> To conclude:
> 
> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
> 3.2 release is our last chance.
> 
> 2) If you afraid that this new behaviour will break too much python 2 code 
> converted with 2to3, we can introduce another 'hasattr' function defined in 
> 2to3 module itself, and make it imported automatically in all files passed 
> through 2to3 transformation pipeline.  This new function will mimic 'hasattr' 
> behaviour from python 2 and converted code should work as expected.


Thanks for the nice analysis and good example.

I disagree with the solution though.  If we want to see the exceptions 
associated 
with actually getting an attribute, then using getattr() instead is a perfectly 
reasonable solution that people can already use without a language change.

But hasattr() has a far different set of use cases, so we should explore
an alternate solution to the problem.  The usual reason that people use
hasattr() instead of getattr() is that they want to check for the presence of
of a method/attribute without actually running it, binding it, or triggering
any other behavior.

As your example shows, property() defeats this intent by actually executing
the code.   A better behavior would not run the code at all.  It would check
the dictionaries along the MRO but not execute any descriptors associated
with a given key.

IMO, this is a much better solution, more in line with known use cases
for hasattr().   If the proposed change when through, it would fail to
address the common use case and cause people to start writing their
own versions of hasattr() that just scan but do not run code.


Raymond


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 23/08/2010 22:47, Raymond Hettinger wrote:

On Aug 23, 2010, at 7:22 AM, Yury Selivanov wrote:


I know the issue has been discussed several times already, however I couldn't 
find any reasonable explanation of its strange behaviour.  The main problem 
with 'hasattr' function is that is swallows all exceptions derived from 
Exception class.  It's a good thing that it doesn't do that with BaseException 
as it was fixed not a long time ago, but it's definitely not enough.

First of all, this behaviour of 'hasattr' contradicts with the very core principle of 
python: "Errors should never pass silently."  And since 'hasattr' function is 
in builtins module and is a widely used function it impacts the whole language.

Secondly, take a look at the following:


class Test:

... @property
... def attr(self):
... self['foo']
...

hasattr(Test(), 'attr')

False

There can be any exception instead of KeyError in the above snippet of code, 
but this small example shows how 'hasattr': misleadingly breaks the code logic 
(1) and masks bug (2).  And that's the simplest possible example, there are 
much more in real life.

While (1) is maybe acceptable for someone, there is no excuse for the (2).  
Moreover, current 'hasattr' behaviour tremendously complicates use of 
'__getattribute__' magic.  And forget about importlib magic with LazyImports, 
one 'hasattr' ruins everything by catching ImportError.


To conclude:

1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
only AttributeError exceptions (exactly like 'getattr').  Probably, Python 3.2 
release is our last chance.

2) If you afraid that this new behaviour will break too much python 2 code 
converted with 2to3, we can introduce another 'hasattr' function defined in 
2to3 module itself, and make it imported automatically in all files passed 
through 2to3 transformation pipeline.  This new function will mimic 'hasattr' 
behaviour from python 2 and converted code should work as expected.


Thanks for the nice analysis and good example.

I disagree with the solution though.  If we want to see the exceptions 
associated
with actually getting an attribute, then using getattr() instead is a perfectly
reasonable solution that people can already use without a language change.

But hasattr() has a far different set of use cases, so we should explore
an alternate solution to the problem.  The usual reason that people use
hasattr() instead of getattr() is that they want to check for the presence of
of a method/attribute without actually running it, binding it, or triggering
any other behavior.

As your example shows, property() defeats this intent by actually executing
the code.   A better behavior would not run the code at all.  It would check
the dictionaries along the MRO but not execute any descriptors associated
with a given key.

IMO, this is a much better solution, more in line with known use cases
for hasattr().   If the proposed change when through, it would fail to
address the common use case and cause people to start writing their
own versions of hasattr() that just scan but do not run code.



It would be backwards incompatible with usage of hasattr for dynamically 
created 'members' using __getattr__ though. For what it's worth I 
*agree* with you [1], but for better or worse hasattr / getattr trigger 
code execution and hasattr can return True for dynamically created 
members. Something to be revisited for Python 4 perhaps.


Michael Foord


[1] A while ago I wrote a couple of blog entries on fetching docstrings 
from members without triggering code execution. It is surprisingly 
convoluted and even my final code had corner cases it didn't handle:

http://www.voidspace.org.uk/python/weblog/arch_d7_2009_05_16.shtml#e1090
http://www.voidspace.org.uk/python/weblog/arch_d7_2009_06_20.shtml#e1103



Raymond


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk



--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Raymond Hettinger :
> Thanks for the nice analysis and good example.
>
> I disagree with the solution though.  If we want to see the exceptions 
> associated
> with actually getting an attribute, then using getattr() instead is a 
> perfectly
> reasonable solution that people can already use without a language change.
>
> But hasattr() has a far different set of use cases, so we should explore
> an alternate solution to the problem.  The usual reason that people use
> hasattr() instead of getattr() is that they want to check for the presence of
> of a method/attribute without actually running it, binding it, or triggering
> any other behavior.

That would break the assumption that:
if hasattr(obj, attr):
 getattr(obj, attr) # won't raise

and hasattr ~=
try:
getattr(obj, attr)
except AttributeError:
return False
else:
return True

>
> As your example shows, property() defeats this intent by actually executing
> the code.   A better behavior would not run the code at all.  It would check
> the dictionaries along the MRO but not execute any descriptors associated
> with a given key.

That doesn't sound to useful to me. A descriptor could be found with
__get__, but that __get__ could just as well raise AttributeError.

>
> IMO, this is a much better solution, more in line with known use cases
> for hasattr().   If the proposed change when through, it would fail to
> address the common use case and cause people to start writing their
> own versions of hasattr() that just scan but do not run code.

Can you provide an example? I've never seen code which explicitly
scans MRO and dicts to avoid triggering code. (Besides
collections.Callable; that's a special case.)



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[P.J. Eby]

At 12:47 PM 8/23/2010 -0700, Raymond Hettinger wrote:

As your example shows, property() defeats this intent by actually executing
the code.   A better behavior would not run the code at all.  It would check
the dictionaries along the MRO but not execute any descriptors associated
with a given key.


That just introduces a new class of error when the descriptor can 
raise AttributeError (e.g. __slots__ descriptors).  And of course, it 
ignoress __getattr__ and __getattribute__.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

On 2010-08-23, at 3:47 PM, Raymond Hettinger wrote:

> 
> On Aug 23, 2010, at 7:22 AM, Yury Selivanov wrote:
> 
>> I know the issue has been discussed several times already, however I 
>> couldn't find any reasonable explanation of its strange behaviour.  The main 
>> problem with 'hasattr' function is that is swallows all exceptions derived 
>> from Exception class.  It's a good thing that it doesn't do that with 
>> BaseException as it was fixed not a long time ago, but it's definitely not 
>> enough.
>> 
>> First of all, this behaviour of 'hasattr' contradicts with the very core 
>> principle of python: "Errors should never pass silently."  And since 
>> 'hasattr' function is in builtins module and is a widely used function it 
>> impacts the whole language.
>> 
>> Secondly, take a look at the following:
>> 
> class Test:
>>   ... @property
>>   ... def attr(self):
>>   ... self['foo']
>>   ... 
> hasattr(Test(), 'attr')
>>   False
>> 
>> There can be any exception instead of KeyError in the above snippet of code, 
>> but this small example shows how 'hasattr': misleadingly breaks the code 
>> logic (1) and masks bug (2).  And that's the simplest possible example, 
>> there are much more in real life.
>> 
>> While (1) is maybe acceptable for someone, there is no excuse for the (2).  
>> Moreover, current 'hasattr' behaviour tremendously complicates use of 
>> '__getattribute__' magic.  And forget about importlib magic with 
>> LazyImports, one 'hasattr' ruins everything by catching ImportError.
>> 
>> 
>> To conclude:
>> 
>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
>> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
>> 3.2 release is our last chance.
>> 
>> 2) If you afraid that this new behaviour will break too much python 2 code 
>> converted with 2to3, we can introduce another 'hasattr' function defined in 
>> 2to3 module itself, and make it imported automatically in all files passed 
>> through 2to3 transformation pipeline.  This new function will mimic 
>> 'hasattr' behaviour from python 2 and converted code should work as expected.
> 
> 
> Thanks for the nice analysis and good example.
> 
> I disagree with the solution though.  If we want to see the exceptions 
> associated 
> with actually getting an attribute, then using getattr() instead is a 
> perfectly 
> reasonable solution that people can already use without a language change.
> 
> But hasattr() has a far different set of use cases, so we should explore
> an alternate solution to the problem.  The usual reason that people use
> hasattr() instead of getattr() is that they want to check for the presence of
> of a method/attribute without actually running it, binding it, or triggering
> any other behavior.
> 
> As your example shows, property() defeats this intent by actually executing
> the code.   A better behavior would not run the code at all.  It would check
> the dictionaries along the MRO but not execute any descriptors associated
> with a given key.
> 
> IMO, this is a much better solution, more in line with known use cases
> for hasattr().   If the proposed change when through, it would fail to
> address the common use case and cause people to start writing their
> own versions of hasattr() that just scan but do not run code.

This is impossible to implement because of '__getattribute__' and '__getattr__' 
methods.  There is no way of detecting the presence of an attribute but trying 
to get it through 'getattr'.  Partial solution like getting information about 
property presence through __dict__ and call __getattribute__/__getattr__ if 
they are defined wouldn't work either.

So, your solution would make 'hasattr' even more incompatible.

-
Yury Selivanov
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 12:47 PM, Raymond Hettinger
 wrote:
>
> On Aug 23, 2010, at 7:22 AM, Yury Selivanov wrote:
>
>> I know the issue has been discussed several times already, however I 
>> couldn't find any reasonable explanation of its strange behaviour.  The main 
>> problem with 'hasattr' function is that is swallows all exceptions derived 
>> from Exception class.  It's a good thing that it doesn't do that with 
>> BaseException as it was fixed not a long time ago, but it's definitely not 
>> enough.
>>
>> First of all, this behaviour of 'hasattr' contradicts with the very core 
>> principle of python: "Errors should never pass silently."  And since 
>> 'hasattr' function is in builtins module and is a widely used function it 
>> impacts the whole language.
>>
>> Secondly, take a look at the following:
>>
> class Test:
>>    ...     @property
>>    ...     def attr(self):
>>    ...         self['foo']
>>    ...
> hasattr(Test(), 'attr')
>>    False
>>
>> There can be any exception instead of KeyError in the above snippet of code, 
>> but this small example shows how 'hasattr': misleadingly breaks the code 
>> logic (1) and masks bug (2).  And that's the simplest possible example, 
>> there are much more in real life.
>>
>> While (1) is maybe acceptable for someone, there is no excuse for the (2).  
>> Moreover, current 'hasattr' behaviour tremendously complicates use of 
>> '__getattribute__' magic.  And forget about importlib magic with 
>> LazyImports, one 'hasattr' ruins everything by catching ImportError.
>>
>>
>> To conclude:
>>
>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to swallow 
>> only AttributeError exceptions (exactly like 'getattr').  Probably, Python 
>> 3.2 release is our last chance.
>>
>> 2) If you afraid that this new behaviour will break too much python 2 code 
>> converted with 2to3, we can introduce another 'hasattr' function defined in 
>> 2to3 module itself, and make it imported automatically in all files passed 
>> through 2to3 transformation pipeline.  This new function will mimic 
>> 'hasattr' behaviour from python 2 and converted code should work as expected.
>
>
> Thanks for the nice analysis and good example.
>
> I disagree with the solution though.  If we want to see the exceptions 
> associated
> with actually getting an attribute, then using getattr() instead is a 
> perfectly
> reasonable solution that people can already use without a language change.
>
> But hasattr() has a far different set of use cases, so we should explore
> an alternate solution to the problem.  The usual reason that people use
> hasattr() instead of getattr() is that they want to check for the presence of
> of a method/attribute without actually running it, binding it, or triggering
> any other behavior.
>
> As your example shows, property() defeats this intent by actually executing
> the code.   A better behavior would not run the code at all.  It would check
> the dictionaries along the MRO but not execute any descriptors associated
> with a given key.
>
> IMO, this is a much better solution, more in line with known use cases
> for hasattr().   If the proposed change when through, it would fail to
> address the common use case and cause people to start writing their
> own versions of hasattr() that just scan but do not run code.

Hm... That sounds like scope creep to me. Properties are supposed to
be cheap and idempotent. Trying to figure out whether an attribute
exists without using __getattr__ is fraught with problems -- as soon
as a class overrides __getattr__ or __getattribute__ you're hosed
anyway.

I can vouch that the reason hasattr() catches too many exceptions is
that when I first added it (around 1990, I think :-) I wasn't very
attuned yet to the problems it could cause.

The main problem I can see with letting exceptions other than
AttributeError bubble through (besides perverted dependencies on the
current semantics) is that there are some situations where it is
pretty arbitrary whether TypeError or AttributeError is raised. I
can't recall the details, and possibly this was more of a problem with
classic classes, but I do think it warrants some research.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 23/08/2010 22:59, Benjamin Peterson wrote:

[snip...]

IMO, this is a much better solution, more in line with known use cases
for hasattr().   If the proposed change when through, it would fail to
address the common use case and cause people to start writing their
own versions of hasattr() that just scan but do not run code.

Can you provide an example? I've never seen code which explicitly
scans MRO and dicts to avoid triggering code. (Besides
collections.Callable; that's a special case.)



The example I linked to in my previous email did exactly that - the use 
case was for finding and displaying docstrings on members in an 
interactive object viewer. We needed to be able to examine objects 
without triggering code execution in them.


To me hasattr *looks* like a passive introspection function, and the 
fact that it can trigger arbitrary code execution is unfortunate - 
especially because a full workaround is pretty arcane.


Michael







--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Michael Foord :
> To me hasattr *looks* like a passive introspection function, and the fact
> that it can trigger arbitrary code execution is unfortunate - especially
> because a full workaround is pretty arcane.

That's the danger of a dynamic language like Python. Even dir() can
now trigger things like that.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Guido van Rossum :
> The main problem I can see with letting exceptions other than
> AttributeError bubble through (besides perverted dependencies on the
> current semantics) is that there are some situations where it is
> pretty arbitrary whether TypeError or AttributeError is raised. I
> can't recall the details, and possibly this was more of a problem with
> classic classes, but I do think it warrants some research.

I believe this was with regards to actual operations, though, not
fetching the attribute. For example, float(a) would raise an
AttributeError for a classic instance and a TypeError for a new-style
instance. However both would raise AttributeError on a.__float__.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Raymond Hettinger]

On Aug 23, 2010, at 1:03 PM, Guido van Rossum wrote:

>> But hasattr() has a far different set of use cases, so we should explore
>> an alternate solution to the problem.  The usual reason that people use
>> hasattr() instead of getattr() is that they want to check for the presence of
>> of a method/attribute without actually running it, binding it, or triggering
>> any other behavior.
>> 
>> As your example shows, property() defeats this intent by actually executing
>> the code.   A better behavior would not run the code at all.  It would check
>> the dictionaries along the MRO but not execute any descriptors associated
>> with a given key.
>> 
>> IMO, this is a much better solution, more in line with known use cases
>> for hasattr().   If the proposed change when through, it would fail to
>> address the common use case and cause people to start writing their
>> own versions of hasattr() that just scan but do not run code.
> 
> Hm... That sounds like scope creep to me. Properties are supposed to
> be cheap and idempotent. Trying to figure out whether an attribute
> exists without using __getattr__ is fraught with problems -- as soon
> as a class overrides __getattr__ or __getattribute__ you're hosed
> anyway.

I don't have a specific proposal in mind.  My main questions are

* Is there anything that hasattr(obj, key) can or should do that
  can't already be done with getattr(obj, key, None)? 
  If not, do we really need to change anything?

* Why do people typically use hasattr() instead getattr()?  
   Aren't they are really trying to just determine 
   whether a key exists somewhere in the MRO?
   If so, then doing anything more than that is probably a surprise.

I know my own uses of hasattr() do not expect any exceptions at all.
It comes up in duck typing support different handling for different 
types of inputs.  If others are using it the same way, I think it is
unlikely that they have unittests to cover the possibility that
hasattr() would ever start raising exceptions.


> I can vouch that the reason hasattr() catches too many exceptions is
> that when I first added it (around 1990, I think :-) I wasn't very
> attuned yet to the problems it could cause.

Fire-up the time machine?


Raymond


P.S.  The current behavior seems to be deeply embedded:

int PyObject_HasAttr(PyObject *o, PyObject *attr_name)
Returns 1 if o has the attribute attr_name, and 0 otherwise. This is equivalent 
to the Python expression hasattr(o, attr_name). This function always succeeds.
int PyObject_HasAttrString(PyObject *o, const char *attr_name)
Returns 1 if o has the attribute attr_name, and 0 otherwise. This is equivalent 
to the Python expression hasattr(o, attr_name). This function always succeeds.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Raymond Hettinger :
> I don't have a specific proposal in mind.  My main questions are
> * Is there anything that hasattr(obj, key) can or should do that
>   can't already be done with getattr(obj, key, None)?
>   If not, do we really need to change anything?

For one, it's annoying when "key" can be None.

> * Why do people typically use hasattr() instead getattr()?
>    Aren't they are really trying to just determine
>    whether a key exists somewhere in the MRO?
>    If so, then doing anything more than that is probably a surprise.

It's generally more convenient that getattr(obj, "blah", None) is not None.

>
> P.S.  The current behavior seems to be deeply embedded:

Well, that's what happens when a behavior is added in 1990. :)



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Raymond Hettinger]

On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:

> 2010/8/23 Michael Foord :
>> To me hasattr *looks* like a passive introspection function, and the fact
>> that it can trigger arbitrary code execution is unfortunate - especially
>> because a full workaround is pretty arcane.

Well said.  The surprise to me in the OP's example was that
the property() was executed.  Regular methods aren't run
by hasattr() so it's hard to remember that when writing code using hasattr().

That is especially unfortunate because someone turning a regular
attribute into a property may be doing so long after client code has
been written (IIRC, that was a key use case for properties).  
IOW, the user of the hasattr() may have had no way
of knowing that an exception could ever be raised
(because it is perfectly safe with regular attributes and methods).


> That's the danger of a dynamic language like Python. Even dir() can
> now trigger things like that.

That's not a honking good thing.
I suggest we don't do more of that.



Raymond

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Raymond Hettinger]

On Aug 23, 2010, at 1:45 PM, Benjamin Peterson wrote:

> 2010/8/23 Raymond Hettinger :
>> 
>> P.S.  The current behavior seems to be deeply embedded:
> 
> Well, that's what happens when a behavior is added in 1990. :)



More generally:  it is an API code smell whenever documentation
promises something like "this always succeeds" ;-)


Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 1:33 PM, Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:
>
> I don't have a specific proposal in mind.
>

That's why I called it scope creep. :-) Trust me, your proposal will not
lead to a quick and better replacement for hasattr(). (See several other
people's replies.)


> My main questions are
>
> * Is there anything that hasattr(obj, key) can or should do that
>   can't already be done with getattr(obj, key, None)?
>   If not, do we really need to change anything?
>

getattr(obj, 'key', None) returns None when obj.key exists and has the value
None. The workaround is ugly.

* Why do people typically use hasattr() instead getattr()?
>Aren't they are really trying to just determine
>whether a key exists somewhere in the MRO?
>If so, then doing anything more than that is probably a surprise.
>

Most users who call hasattr() probably don't even know what MRO means. They
call hasattr() because they want to avoid a try/except clause. The reasons
they are not calling getattr(obj, key, None) could be many: never heard of
it, too obscure (it surely doesn't spell "has the attribute" like hasattr()
does to the beginning user), or (probably common) the actual attribute
access is in some other piece of code they are about to call but don't
control.


> I know my own uses of hasattr() do not expect any exceptions at all.
> It comes up in duck typing support different handling for different
> types of inputs.  If others are using it the same way, I think it is
> unlikely that they have unittests to cover the possibility that
> hasattr() would ever start raising exceptions.
>

It already raises *some* exceptions (those derived from BaseException but
not Exception). I think that change was a definite non-event.


> I can vouch that the reason hasattr() catches too many exceptions is
> that when I first added it (around 1990, I think :-) I wasn't very
> attuned yet to the problems it could cause.
>
>
> Fire-up the time machine?
>
>
> Raymond
>
>
> P.S.  The current behavior seems to be deeply embedded:
>

But note that hasattr() doesn't call those.

> int PyObject_HasAttr(PyObject * *o*,
> PyObject * 
> *attr_name*)<#12aa0a7c7d265b64_PyObject_HasAttr>
> Returns 1 if *o* has the attribute *attr_name*, and 0 otherwise. This is
> equivalent to the Python expression hasattr(o, attr_name). This function
> always succeeds. int 
> PyObject_HasAttrString(PyObject
> * *o*, const char* *attr_name*) <#12aa0a7c7d265b64_PyObject_HasAttrString>
> Returns 1 if *o* has the attribute *attr_name*, and 0 otherwise. This is
> equivalent to the Python expression hasattr(o, attr_name). This function
> always succeeds.
>
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

On 2010-08-23, at 4:33 PM, Raymond Hettinger wrote:
> 
> On Aug 23, 2010, at 1:03 PM, Guido van Rossum wrote:
> 
>>> But hasattr() has a far different set of use cases, so we should explore
>>> an alternate solution to the problem.  The usual reason that people use
>>> hasattr() instead of getattr() is that they want to check for the presence 
>>> of
>>> of a method/attribute without actually running it, binding it, or triggering
>>> any other behavior.
>>> 
>>> As your example shows, property() defeats this intent by actually executing
>>> the code.   A better behavior would not run the code at all.  It would check
>>> the dictionaries along the MRO but not execute any descriptors associated
>>> with a given key.
>>> 
>>> IMO, this is a much better solution, more in line with known use cases
>>> for hasattr().   If the proposed change when through, it would fail to
>>> address the common use case and cause people to start writing their
>>> own versions of hasattr() that just scan but do not run code.
>> 
>> Hm... That sounds like scope creep to me. Properties are supposed to
>> be cheap and idempotent. Trying to figure out whether an attribute
>> exists without using __getattr__ is fraught with problems -- as soon
>> as a class overrides __getattr__ or __getattribute__ you're hosed
>> anyway.
> 
> I don't have a specific proposal in mind.  My main questions are
> 
> * Is there anything that hasattr(obj, key) can or should do that
>   can't already be done with getattr(obj, key, None)? 
>   If not, do we really need to change anything?

OK, if my key has None value, how will you detect it with 'getattr'?

... marker = object()
... if (getattr(obj, key, marker) is not marker:

Like in the above?  Too much code.  Ugly code.  Just to hide the issue
under the carpet.

> * Why do people typically use hasattr() instead getattr()?  
>Aren't they are really trying to just determine 
>whether a key exists somewhere in the MRO?
>If so, then doing anything more than that is probably a surprise.

My example in the initial email clearly showed that sometimes, key is 
in MRO, but 'hasattr' returns False.

For everything in any complicated system (which Python obviously is)
should be a strict protocol.  In case of properties this protocol is to
raise AttributeError if property is missing.

Everything else is a potential bug.  Current 'hasattr' just masks them.

-
Yury

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 23/08/2010 23:13, Benjamin Peterson wrote:

2010/8/23 Michael Foord:

To me hasattr *looks* like a passive introspection function, and the fact
that it can trigger arbitrary code execution is unfortunate - especially
because a full workaround is pretty arcane.

That's the danger of a dynamic language like Python. Even dir() can
now trigger things like that.



Well yes. One of the reasons a full workaround is so arcane is that if 
you *really* don't want to trigger code execution you can't call dir...


(Otherwise name in dir(obj) is a reasonable approximation for 
hasattr(obj, name).)


Michael

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Raymond Hettinger :
>
> On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:
>
>> 2010/8/23 Michael Foord :
>>> To me hasattr *looks* like a passive introspection function, and the fact
>>> that it can trigger arbitrary code execution is unfortunate - especially
>>> because a full workaround is pretty arcane.

hasattr(x, "y") doesn't look any more passive to me the x.y.

>
> Well said.  The surprise to me in the OP's example was that
> the property() was executed.  Regular methods aren't run
> by hasattr() so it's hard to remember that when writing code using hasattr().

Hard to remember compared to what?

>
> That is especially unfortunate because someone turning a regular
> attribute into a property may be doing so long after client code has
> been written (IIRC, that was a key use case for properties).
> IOW, the user of the hasattr() may have had no way
> of knowing that an exception could ever be raised
> (because it is perfectly safe with regular attributes and methods).

Better to raise an exception into unexpecting code that to have subtly
different lookup rules between getattr and hasattr.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 1:50 PM, Raymond Hettinger
 wrote:
>
> On Aug 23, 2010, at 1:45 PM, Benjamin Peterson wrote:
>
> > 2010/8/23 Raymond Hettinger :
> >>
> >> P.S.  The current behavior seems to be deeply embedded:
> >
> > Well, that's what happens when a behavior is added in 1990. :)
>
> More generally:  it is an API code smell whenever documentation
> promises something like "this always succeeds"     ;-)

Changing C APIs is even harder than changing Python API because you
can't add exceptions to something that wasn't returning exceptions
before. We did that for comparisons in the past and it was a pain (but
worth it). For these two little APIs I think it is not worth it,
though it may be worth it to create new APIs that *do* return
exceptions.

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Guido van Rossum :
> Changing C APIs is even harder than changing Python API because you
> can't add exceptions to something that wasn't returning exceptions
> before. We did that for comparisons in the past and it was a pain (but
> worth it). For these two little APIs I think it is not worth it,
> though it may be worth it to create new APIs that *do* return
> exceptions.

+1 I propose we add PyObject_HasattrWithErrors to parallel
PyDict_GetItemWithErrors.


-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 23/08/2010 23:55, Benjamin Peterson wrote:

2010/8/23 Raymond Hettinger:

On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:


2010/8/23 Michael Foord:

To me hasattr *looks* like a passive introspection function, and the fact
that it can trigger arbitrary code execution is unfortunate - especially
because a full workaround is pretty arcane.

hasattr(x, "y") doesn't look any more passive to me the x.y.
One says "does this object have attribute y" the other fetches attribute 
y. I'm amazed they don't look different to you. Given Python's object 
model there is no reason that the first *should* fetch the attribute to 
determine that it is present, *except* for the dynamic attribute 
creation of __getattr__ and __getattribute__.


For properties there is *no reason* why code should be executed merely 
in order to discover if the attribute exists or not.


I'm in both camps though. As we *are* triggering code execution I don't 
think we should mask exceptions. I just wish we weren't triggering code 
execution unnecessarily.


Michael



Well said.  The surprise to me in the OP's example was that
the property() was executed.  Regular methods aren't run
by hasattr() so it's hard to remember that when writing code using hasattr().

Hard to remember compared to what?


That is especially unfortunate because someone turning a regular
attribute into a property may be doing so long after client code has
been written (IIRC, that was a key use case for properties).
IOW, the user of the hasattr() may have had no way
of knowing that an exception could ever be raised
(because it is perfectly safe with regular attributes and methods).

Better to raise an exception into unexpecting code that to have subtly
different lookup rules between getattr and hasattr.





--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Michael Foord :
>  On 23/08/2010 23:55, Benjamin Peterson wrote:
>>
>> 2010/8/23 Raymond Hettinger:
>>>
>>> On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:
>>>
 2010/8/23 Michael Foord:
>
> To me hasattr *looks* like a passive introspection function, and the
> fact
> that it can trigger arbitrary code execution is unfortunate -
> especially
> because a full workaround is pretty arcane.
>>
>> hasattr(x, "y") doesn't look any more passive to me the x.y.
>
> One says "does this object have attribute y" the other fetches attribute y.
> I'm amazed they don't look different to you. Given Python's object model
> there is no reason that the first *should* fetch the attribute to determine
> that it is present, *except* for the dynamic attribute creation of
> __getattr__ and __getattribute__.

Actually, I'd say given Python's object model, you have to have the
attribute in hand to determine that it is present.

>
> For properties there is *no reason* why code should be executed merely in
> order to discover if the attribute exists or not.

Properties are allowed to raise AttributeError, so you technically
have to execute it.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 24/08/2010 00:05, Benjamin Peterson wrote:

2010/8/23 Michael Foord:

  On 23/08/2010 23:55, Benjamin Peterson wrote:

2010/8/23 Raymond Hettinger:

On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:


2010/8/23 Michael Foord:

To me hasattr *looks* like a passive introspection function, and the
fact
that it can trigger arbitrary code execution is unfortunate -
especially
because a full workaround is pretty arcane.

hasattr(x, "y") doesn't look any more passive to me the x.y.

One says "does this object have attribute y" the other fetches attribute y.
I'm amazed they don't look different to you. Given Python's object model
there is no reason that the first *should* fetch the attribute to determine
that it is present, *except* for the dynamic attribute creation of
__getattr__ and __getattribute__.

Actually, I'd say given Python's object model, you have to have the
attribute in hand to determine that it is present.


For properties there is *no reason* why code should be executed merely in
order to discover if the attribute exists or not.

Properties are allowed to raise AttributeError, so you technically
have to execute it.

Properties are allowed to do whatever the heck they want. That doesn't 
mean that you have to execute code to determine whether they exist or not.


If fetching an attribute raises an AttributeError it doesn't mean that 
attribute doesn't exist (although I admit that at the moment this is 
exactly what hasattr uses to mean) it just means that fetching that 
attribute raised an AttributeError. Even if you allow other exceptions 
to propagate you are still left with the fact that an AttributeError 
raised as a bug will still be silenced and interpreted as meaning that 
hasattr should just return False.


Michael Foord

--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Michael Foord :
> Properties are allowed to do whatever the heck they want. That doesn't mean
> that you have to execute code to determine whether they exist or not.

I thought you were trying to determine whether the attribute exists
not the property.
>
> If fetching an attribute raises an AttributeError it doesn't mean that
> attribute doesn't exist (although I admit that at the moment this is exactly
> what hasattr uses to mean) it just means that fetching that attribute raised
> an AttributeError. Even if you allow other exceptions to propagate you are
> still left with the fact that an AttributeError raised as a bug will still
> be silenced and interpreted as meaning that hasattr should just return
> False.

Raised as a bug? Is this not a valid pattern?

@property
def myprop(self):
if not self.myprop_support:
raise AttributeError("myprop")



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

On 2010-08-23, at 5:02 PM, Michael Foord wrote:

> On 23/08/2010 23:55, Benjamin Peterson wrote:
>> 2010/8/23 Raymond Hettinger:
>>> On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:
>>> 
 2010/8/23 Michael Foord:
> To me hasattr *looks* like a passive introspection function, and the fact
> that it can trigger arbitrary code execution is unfortunate - especially
> because a full workaround is pretty arcane.
>> hasattr(x, "y") doesn't look any more passive to me the x.y.
> One says "does this object have attribute y" the other fetches attribute y. 
> I'm amazed they don't look different to you. Given Python's object model 
> there is no reason that the first *should* fetch the attribute to determine 
> that it is present, *except* for the dynamic attribute creation of 
> __getattr__ and __getattribute__.

As I understand the only possible way to make 'hasattr' work as it name 
indicates (i.e. just check if attribute exists, not run it), is to add another 
magic method(s?) to the existing __getattr__ and __getattribute__ which will 
tell whether attribute exists or not, and by default this method would mimic 
current 'hasattr' behaviour.

-
Yury

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

On 2010-08-23, at 5:22 PM, Yury Selivanov wrote:

> On 2010-08-23, at 5:02 PM, Michael Foord wrote:
> 
>> On 23/08/2010 23:55, Benjamin Peterson wrote:
>>> 2010/8/23 Raymond Hettinger:
 On Aug 23, 2010, at 1:13 PM, Benjamin Peterson wrote:
 
> 2010/8/23 Michael Foord:
>> To me hasattr *looks* like a passive introspection function, and the fact
>> that it can trigger arbitrary code execution is unfortunate - especially
>> because a full workaround is pretty arcane.
>>> hasattr(x, "y") doesn't look any more passive to me the x.y.
>> One says "does this object have attribute y" the other fetches attribute y. 
>> I'm amazed they don't look different to you. Given Python's object model 
>> there is no reason that the first *should* fetch the attribute to determine 
>> that it is present, *except* for the dynamic attribute creation of 
>> __getattr__ and __getattribute__.
> 
> As I understand the only possible way to make 'hasattr' work as it name 
> indicates (i.e. just check if attribute exists, not run it), is to add 
> another magic method(s?) to the existing __getattr__ and __getattribute__ 
> which will tell whether attribute exists or not, and by default this method 
> would mimic current 'hasattr' behaviour.

I'm not sure we should do this, but if such method exists, let's call it 
__hasattribute__, ORMs, for instance, probably would benefit.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Nick Coghlan]

> Properties are allowed to do whatever the heck they want. That doesn't mean
> that you have to execute code to determine whether they exist or not.

If you don't want to execute properties, do the lookup on the type,
not the instance (obviously, you know the dance you need to do, since
you've linked the code where you did it). Having apparently simple
query operations like hasattr/getattr/len/etc execute arbitrary code
is where much of the language's flexibility comes from, so I don't see
how it can really be surprising when it happens.

To me, Python's definition of an object having an attribute is "Object
x has an attribute y if x.y does not raise AttributeError". That means
it can't figure out whether or not the attribute exists without
actually attempting to retrieve it. There are a few places where we
instead use a heuristic that says an attribute *probably* exists if it
appears in the instance dictionary, or in the dictionary of one of the
types in the MRO, and magic methods have the rule that they must be
defined on the type rather than the instance in order to count from
the interpreter's point of view, but neither of those things change
the basic definition.

For the record, +1 on narrowing the scope of the exception suppression
in hasattr() to only AttributeError, and adding new C API functions
that expose the new behaviour. (I've actually long assumed that
AttributeError *was* the only thing suppressed by hasattr()).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Michael Foord]

 On 24/08/2010 00:40, Nick Coghlan wrote:

Properties are allowed to do whatever the heck they want. That doesn't mean
that you have to execute code to determine whether they exist or not.

If you don't want to execute properties, do the lookup on the type,
not the instance (obviously, you know the dance you need to do, since
you've linked the code where you did it). Having apparently simple
query operations like hasattr/getattr/len/etc execute arbitrary code
is where much of the language's flexibility comes from, so I don't see
how it can really be surprising when it happens.


Certainly that is true for len. getattr obviously involves invoking code 
if you are fetching a property or descriptor. No idea how you conclude 
that hasattr executing code adds flexibility to the language though.


Yes I know the dance (walking the mro fetching the attribute out of the 
appropriate type __dict__ or the instance dict - or looking on the 
metaclass if the object you are introspecting is a type itself), it is 
just not trivial - which is why I think it is a shame that people are 
forced to implement it just to ask if a member exists without triggering 
code execution.



To me, Python's definition of an object having an attribute is "Object
x has an attribute y if x.y does not raise AttributeError".


Right, and to me Python's object model (the lookup rules hinted at 
above) define whether or not an object "has an attribute" or not. We 
just disagree on this.


However, it is irrelevant so not really worth continuing the discussion. 
Changing hasattr in this way would be backwards incompatible and so 
cannot be done. Doesn't mean I have to like it though. :-)


Michael


That means
it can't figure out whether or not the attribute exists without
actually attempting to retrieve it. There are a few places where we
instead use a heuristic that says an attribute *probably* exists if it
appears in the instance dictionary, or in the dictionary of one of the
types in the MRO, and magic methods have the rule that they must be
defined on the type rather than the instance in order to count from
the interpreter's point of view, but neither of those things change
the basic definition.

For the record, +1 on narrowing the scope of the exception suppression
in hasattr() to only AttributeError, and adding new C API functions
that expose the new behaviour. (I've actually long assumed that
AttributeError *was* the only thing suppressed by hasattr()).

Cheers,
Nick.




--
http://www.ironpythoninaction.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Michael Foord :
>  On 24/08/2010 00:40, Nick Coghlan wrote:
>>>
>>> Properties are allowed to do whatever the heck they want. That doesn't
>>> mean
>>> that you have to execute code to determine whether they exist or not.
>>
>> If you don't want to execute properties, do the lookup on the type,
>> not the instance (obviously, you know the dance you need to do, since
>> you've linked the code where you did it). Having apparently simple
>> query operations like hasattr/getattr/len/etc execute arbitrary code
>> is where much of the language's flexibility comes from, so I don't see
>> how it can really be surprising when it happens.
>
> Certainly that is true for len. getattr obviously involves invoking code if
> you are fetching a property or descriptor. No idea how you conclude that
> hasattr executing code adds flexibility to the language though.
>
> Yes I know the dance (walking the mro fetching the attribute out of the
> appropriate type __dict__ or the instance dict - or looking on the metaclass
> if the object you are introspecting is a type itself), it is just not
> trivial - which is why I think it is a shame that people are forced to
> implement it just to ask if a member exists without triggering code
> execution.

Sounds like something for a new function.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 2:22 PM, Yury Selivanov  wrote:
> As I understand the only possible way to make 'hasattr' work as it name 
> indicates (i.e. just check if attribute exists, not run it), is to add 
> another magic method(s?) to the existing __getattr__ and __getattribute__ 
> which will tell whether attribute exists or not, and by default this method 
> would mimic current 'hasattr' behaviour.

You nailed it. At the lowest level (either in C or in Python) there is
no way to check for an attribute's presence without getting its value.
This has so far been fundamental, and everything else is just
appearances.

I propose that this is good enough and that we should at this point
not try to invent another protocol, just decide whether hasattr() can
be fixed (and leaving PyObject_HasAttr*() alone).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

> On 2010-08-23, at 6:00 PM, Guido van Rossum wrote:
> On Mon, Aug 23, 2010 at 2:22 PM, Yury Selivanov  wrote:
>> As I understand the only possible way to make 'hasattr' work as it name 
>> indicates (i.e. just check if attribute exists, not run it), is to add 
>> another magic method(s?) to the existing __getattr__ and __getattribute__ 
>> which will tell whether attribute exists or not, and by default this method 
>> would mimic current 'hasattr' behaviour.
> 
> You nailed it. At the lowest level (either in C or in Python) there is
> no way to check for an attribute's presence without getting its value.
> This has so far been fundamental, and everything else is just
> appearances.
> 
> I propose that this is good enough and that we should at this point
> not try to invent another protocol, just decide whether hasattr() can
> be fixed (and leaving PyObject_HasAttr*() alone).

BTW, is it possible to add new magic method __hasattr__?  Maybe not
in Python 3.2, but in general.

The more I think about it the more I like the idea.  By default, 'hasattr'
would check MRO for the attribute, if not found - check for __hasattr__,
if not found - fallback to the current schema with 'getattr'.

This would not break any current code, but open a lot of potential 
(especially for things that implement some lazy loading protocols).

>From the performance point of view, I believe it shouldn't change much,
as current 'hasattr' which uses 'getattr' which calls __getattr(ibute)__ 
methods.

I know this is much more then just fixing exception swallowing, but this
is somewhat required functionality to complete the current dynamism 
python offers.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Nick Coghlan]

On Tue, Aug 24, 2010 at 7:40 AM, Nick Coghlan  wrote:
>> Properties are allowed to do whatever the heck they want. That doesn't mean
>> that you have to execute code to determine whether they exist or not.
>
> If you don't want to execute properties, do the lookup on the type,
> not the instance (obviously, you know the dance you need to do, since
> you've linked the code where you did it). Having apparently simple
> query operations like hasattr/getattr/len/etc execute arbitrary code
> is where much of the language's flexibility comes from, so I don't see
> how it can really be surprising when it happens.

Now, it may be worth considering an addition to the inspect module
that was basically:

def getattr_static(obj, attr):
"""Retrieve attributes without triggering dynamic lookup via the
descriptor protocol,
__getattr__ or __getattribute__.

Note: this function may not be able to retrieve all attributes
reported by dir(obj)
"""
try:
instance_dict = object.__getattribute__(obj, "__dict__")
except AttributeError:
pass
else:
if attr in instance_dict:
return instance_dict[attr]
for entry in getmro(obj.__class__):
try:
return entry.__dict__[attr]
except AttributeError:
pass

(not needing to deal with classic classes simplifies things a bit)

So, allowing for the fact that dir() may report attributes that can
only be found via dynamic lookup, your get_docstrings example could
become something like:

def get_docstrings(obj):
try:
members = dir(obj)
except Exception:
members = []
for member in members:
try:
doc = getattr_static(obj, member).__doc__
except AttributeError:
doc = None
yield member, doc

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Yury Selivanov :
>> On 2010-08-23, at 6:00 PM, Guido van Rossum wrote:
>> On Mon, Aug 23, 2010 at 2:22 PM, Yury Selivanov  wrote:
> BTW, is it possible to add new magic method __hasattr__?  Maybe not
> in Python 3.2, but in general.

-1 The whole point of hasattr() is that getattr() on that attribute
would return something. Let's not destroy that.

-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Nick Coghlan]

On Tue, Aug 24, 2010 at 7:49 AM, Michael Foord
 wrote:
> Certainly that is true for len. getattr obviously involves invoking code if
> you are fetching a property or descriptor. No idea how you conclude that
> hasattr executing code adds flexibility to the language though.

Proxy objects like the one in the weakref module don't work if
hasattr() doesn't implicitly invoke __getattr__.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

> On 2010-08-23, at 6:17 PM, Benjamin Peterson wrote:
> 2010/8/23 Yury Selivanov :
>>> On 2010-08-23, at 6:00 PM, Guido van Rossum wrote:
>>> On Mon, Aug 23, 2010 at 2:22 PM, Yury Selivanov  
>>> wrote:
>> BTW, is it possible to add new magic method __hasattr__?  Maybe not
>> in Python 3.2, but in general.
> 
> -1 The whole point of hasattr() is that getattr() on that attribute
> would return something. Let's not destroy that.

By default, if you don't overload __hasattr__, it will do exactly that.

But let's imagine the situation in some ORM.  Sometimes, engine knows that
attribute exists even before executing it (which leads to querying DB), and
thus it is possible to return True in the overloaded __hasattr__.

Example:
... if hasattr(entity, 'title'): # <- __hasattr__ has the info in metadata,
 #no query to DB
... title = entity.title # <- now, we can query the DB

So, the proposed magic method is not intended to change the protocol,
but to complement and enhance it.

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Yury Selivanov :
>> On 2010-08-23, at 6:17 PM, Benjamin Peterson wrote:
>> 2010/8/23 Yury Selivanov :
 On 2010-08-23, at 6:00 PM, Guido van Rossum wrote:
 On Mon, Aug 23, 2010 at 2:22 PM, Yury Selivanov  
 wrote:
>>> BTW, is it possible to add new magic method __hasattr__?  Maybe not
>>> in Python 3.2, but in general.
>>
>> -1 The whole point of hasattr() is that getattr() on that attribute
>> would return something. Let's not destroy that.
>
> By default, if you don't overload __hasattr__, it will do exactly that.
>
> But let's imagine the situation in some ORM.  Sometimes, engine knows that
> attribute exists even before executing it (which leads to querying DB), and
> thus it is possible to return True in the overloaded __hasattr__.
>
> Example:
> ... if hasattr(entity, 'title'): # <- __hasattr__ has the info in metadata,
>                                 #    no query to DB
> ...     title = entity.title     # <- now, we can query the DB

In this case, I think it would make more sense to ask the model:
type(entity).has_property("title")

>
> So, the proposed magic method is not intended to change the protocol,
> but to complement and enhance it.

But it still raises the potential to break the relationship between
hasattr and getattr.

I think this should require a PEP.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Nick Coghlan]

On Tue, Aug 24, 2010 at 8:15 AM, Nick Coghlan  wrote:
> Now, it may be worth considering an addition to the inspect module
> that was basically:
>
> def getattr_static(obj, attr):
>    """Retrieve attributes without triggering dynamic lookup via the
> descriptor protocol,
>        __getattr__ or __getattribute__.
>
>        Note: this function may not be able to retrieve all attributes
> reported by dir(obj)
>    """
>    try:
>        instance_dict = object.__getattribute__(obj, "__dict__")
>    except AttributeError:
>        pass
>    else:
>        if attr in instance_dict:
>            return instance_dict[attr]
>    for entry in getmro(obj.__class__):
>        try:
>            return entry.__dict__[attr]
>        except AttributeError:
>            pass

Second attempt with a default value parameter and correctly raising
AttributeError if not found:

_sentinel = object()
def getattr_static(obj, attr, default=_sentinel):
"""Retrieve attributes without triggering dynamic lookup via the
descriptor protocol,  __getattr__ or __getattribute__.

Note: this function may not be able to retrieve all attributes
reported by dir(obj)
"""
try:
instance_dict = object.__getattribute__(obj, "__dict__")
except AttributeError:
pass
else:
if attr in instance_dict:
return instance_dict[attr]
for entry in getmro(obj.__class__):
try:
return entry.__dict__[attr]
except AttributeError:
pass
if default is not _sentinel:
return default
raise AttributeError(attr)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[P.J. Eby]

At 12:02 AM 8/24/2010 +0300, Michael Foord wrote:
For properties there is *no reason* why code should be executed 
merely in order to discover if the attribute exists or not.


That depends on what you mean by "exists".  Note that a property 
might raise AttributeError to signal that the attribute is not 
currently set.  Likewise, unless you special case __slots__ 
descriptors, you can have the bizarre condition where hasattr() will 
return True, but getattr() will still raise an AttributeError.


The idea that you could determine the presence of an attribute on an 
object without executing that object's code is something that hasn't 
been practical since the birth of descriptors in Python 2.2.



Yes I know the dance (walking the mro fetching the attribute out of 
the appropriate type __dict__ or the instance dict - or looking on 
the metaclass if the object you are introspecting is a type itself), 
it is just not trivial - which is why I think it is a shame that 
people are forced to implement it just to ask if a member exists 
without triggering code execution.


Even if you implement it, you will get wrong answers in some 
cases.  __getattribute__ is allowed to throw out the entire algorithm 
you just described and replace it utterly with something else.


My ProxyTypes library makes use of that fact, for example, so if you 
actually attempted to inspect a proxy instance with your 
re-implemented "dance", your code will fail to notice what attributes 
the proxy actually has.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Nick Coghlan]

On Tue, Aug 24, 2010 at 8:25 AM, Benjamin Peterson  wrote:
> 2010/8/23 Yury Selivanov :
>> So, the proposed magic method is not intended to change the protocol,
>> but to complement and enhance it.
>
> But it still raises the potential to break the relationship between
> hasattr and getattr.
>
> I think this should require a PEP.

Definitely needs a PEP, and will require some solid use cases to
explain why allowing optimisation of hasattr() is the right way to go.
Complexity isn't free, and I doubt the gains here will justify the
costs, but that's one of the things the PEP process is intended to
figure out.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[P.J. Eby]

At 06:12 PM 8/23/2010 -0400, Yury Selivanov wrote:

BTW, is it possible to add new magic method __hasattr__?  Maybe not
in Python 3.2, but in general.


In order to do this properly, you'd need to also add __has__ or 
__exists__ (or some such) to the descriptor protocol; otherwise you 
break descriptors' ability to operate independently of the class 
they're used in.  You would probably also need a __hasattribute__, in 
order to be able to properly synchronize with __getattr__/__getattribute__.


Seems like overkill to me, though, as I'm not sure how such a 
protocol actually helps ORM or persistence schemes (and I've written 
a few).  Pretty much, if you're trying to check for the existence of 
an attribute, you're probably about to be getting that attribute 
anyway.  (i.e. why query the existence of an attribute you *don't* 
intend to use?)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Barry Warsaw]

On Aug 23, 2010, at 03:45 PM, Benjamin Peterson wrote:

>It's generally more convenient that getattr(obj, "blah", None) is not
>None.

Clearly, the solution is a new builtin called 'Missing'.


-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 3:45 PM, P.J. Eby  wrote:
> At 06:12 PM 8/23/2010 -0400, Yury Selivanov wrote:
>>
>> BTW, is it possible to add new magic method __hasattr__?  Maybe not
>> in Python 3.2, but in general.
>
> In order to do this properly, you'd need to also add __has__ or __exists__
> (or some such) to the descriptor protocol; otherwise you break descriptors'
> ability to operate independently of the class they're used in.  You would
> probably also need a __hasattribute__, in order to be able to properly
> synchronize with __getattr__/__getattribute__.
>
> Seems like overkill to me, though, as I'm not sure how such a protocol
> actually helps ORM or persistence schemes (and I've written a few).  Pretty
> much, if you're trying to check for the existence of an attribute, you're
> probably about to be getting that attribute anyway.  (i.e. why query the
> existence of an attribute you *don't* intend to use?)

Right. This sounds like way too big a gun to kill this particular
mosquito. If Yury wants to write a PEP I won't stop him, but I expect
that it will be rejected for want of important use cases compared to
the complexity of the solution. There just are too many places that
would be affected, for too little value.

So just be warned.

OTOH I still think that fixing hasattr() to be mroe like getattr(obj,
key, None) has a high value and a relative low risk.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Greg Ewing]

Michael Foord wrote:

It would be backwards incompatible with usage of hasattr for dynamically 
created 'members' using __getattr__ though.


Also keep in mind that builtin types mostly don't keep
their attributes in dictionaries. To make this work
properly, hasattr would need its own special method.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Steven D'Aprano]

On Tue, 24 Aug 2010 06:50:19 am Guido van Rossum wrote:

> > * Is there anything that hasattr(obj, key) can or should do that
> >   can't already be done with getattr(obj, key, None)?
> >   If not, do we really need to change anything?
>
> getattr(obj, 'key', None) returns None when obj.key exists and has
> the value None. The workaround is ugly.

Why do you say it's ugly? It's a short, sweet, simple two-liner:

mark = object()
getattr(obj, 'key', mark) is not mark

Nothing ugly about it at all. But if somebody really objected to using a 
two lines, they could put it in a utility function.

It still doesn't cope with dynamically-generated attributes that are 
either expensive or have side-effects (both of which are probably poor 
design, but nevertheless I'm sure they're out there), but neither does 
the existing hasattr.


> * Why do people typically use hasattr() instead getattr()?
>
> >Aren't they are really trying to just determine
> >whether a key exists somewhere in the MRO?
> >If so, then doing anything more than that is probably a
> > surprise.
>
> Most users who call hasattr() probably don't even know what MRO
> means. 

Well, yes, but most users never write __getattr__ or __getattribute__ 
methods either.

I have always thought that hasattr() does what it says on the box: it 
tests for the *existence* of an attribute, that is, one that statically 
exists rather than being dynamically generated. In other words, it is a 
key in the instance __dict__ or is inherited from the class __dict__ or 
that of a superclass, or a __slot__.

Now that I know that hasattr doesn't do what I thought it does or what 
the name implies it does, it has little or no utility for me. In the 
future, I'll just write a try...except block and catch errors if the 
attribute doesn't exist.



-- 
Steven D'Aprano
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Steven D'Aprano :
> On Tue, 24 Aug 2010 06:50:19 am Guido van Rossum wrote:
>
>> > * Is there anything that hasattr(obj, key) can or should do that
>> >   can't already be done with getattr(obj, key, None)?
>> >   If not, do we really need to change anything?
>>
>> getattr(obj, 'key', None) returns None when obj.key exists and has
>> the value None. The workaround is ugly.
>
> Why do you say it's ugly? It's a short, sweet, simple two-liner:
>
> mark = object()
> getattr(obj, 'key', mark) is not mark
>
> Nothing ugly about it at all. But if somebody really objected to using a
> two lines, they could put it in a utility function.

That's not all, though. You still have to test it with an "if" and
that gets cumbersome after a while.

>
> I have always thought that hasattr() does what it says on the box: it
> tests for the *existence* of an attribute, that is, one that statically
> exists rather than being dynamically generated. In other words, it is a
> key in the instance __dict__ or is inherited from the class __dict__ or
> that of a superclass, or a __slot__.
>
> Now that I know that hasattr doesn't do what I thought it does or what
> the name implies it does, it has little or no utility for me. In the
> future, I'll just write a try...except block and catch errors if the
> attribute doesn't exist.

Well, that's exactly what hasattr does...



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Brett Cannon]

On Mon, Aug 23, 2010 at 17:04, Benjamin Peterson  wrote:
> 2010/8/23 Steven D'Aprano :
>> On Tue, 24 Aug 2010 06:50:19 am Guido van Rossum wrote:
>>
>>> > * Is there anything that hasattr(obj, key) can or should do that
>>> >   can't already be done with getattr(obj, key, None)?
>>> >   If not, do we really need to change anything?
>>>
>>> getattr(obj, 'key', None) returns None when obj.key exists and has
>>> the value None. The workaround is ugly.
>>
>> Why do you say it's ugly? It's a short, sweet, simple two-liner:
>>
>> mark = object()
>> getattr(obj, 'key', mark) is not mark
>>
>> Nothing ugly about it at all. But if somebody really objected to using a
>> two lines, they could put it in a utility function.
>
> That's not all, though. You still have to test it with an "if" and
> that gets cumbersome after a while.

It is also non-obvious to any beginner. Are we really going to want to
propagate the knowledge of this trick as a fundamental idiom? I would
rather leave hasattr in that instance. But I'm +1 on only swallowing
AttributeError.

-Brett

>
>>
>> I have always thought that hasattr() does what it says on the box: it
>> tests for the *existence* of an attribute, that is, one that statically
>> exists rather than being dynamically generated. In other words, it is a
>> key in the instance __dict__ or is inherited from the class __dict__ or
>> that of a superclass, or a __slot__.
>>
>> Now that I know that hasattr doesn't do what I thought it does or what
>> the name implies it does, it has little or no utility for me. In the
>> future, I'll just write a try...except block and catch errors if the
>> attribute doesn't exist.
>
> Well, that's exactly what hasattr does...
>
>
>
> --
> Regards,
> Benjamin
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

On Mon, Aug 23, 2010 at 4:56 PM, Steven D'Aprano  wrote:
> On Tue, 24 Aug 2010 06:50:19 am Guido van Rossum wrote:
>
>> > * Is there anything that hasattr(obj, key) can or should do that
>> >   can't already be done with getattr(obj, key, None)?
>> >   If not, do we really need to change anything?
>>
>> getattr(obj, 'key', None) returns None when obj.key exists and has
>> the value None. The workaround is ugly.
>
> Why do you say it's ugly? It's a short, sweet, simple two-liner:
>
> mark = object()
> getattr(obj, 'key', mark) is not mark
>
> Nothing ugly about it at all. But if somebody really objected to using a
> two lines, they could put it in a utility function.

Because if you didn't have the foresight to write that utility
function, you have to break your train of thought (aka yak shaving) to
either write it (big yak) or write those two lines (little yak, times
the number of times it happens). Whereas with hasattr() you can just
type the correct hasattr() expression in-line in the if that you
already started typing.

> It still doesn't cope with dynamically-generated attributes that are
> either expensive or have side-effects (both of which are probably poor
> design, but nevertheless I'm sure they're out there), but neither does
> the existing hasattr.
>
>
>> * Why do people typically use hasattr() instead getattr()?
>>
>> >    Aren't they are really trying to just determine
>> >    whether a key exists somewhere in the MRO?
>> >    If so, then doing anything more than that is probably a
>> > surprise.
>>
>> Most users who call hasattr() probably don't even know what MRO
>> means.
>
> Well, yes, but most users never write __getattr__ or __getattribute__
> methods either.
>
> I have always thought that hasattr() does what it says on the box: it
> tests for the *existence* of an attribute, that is, one that statically
> exists rather than being dynamically generated. In other words, it is a
> key in the instance __dict__ or is inherited from the class __dict__ or
> that of a superclass, or a __slot__.

It tests for the existence of an attribute -- how the attribute is
defined should not have to occur to you (and there are lots of other
ways for attributes to be defined besides the ways you came up with
just now).

> Now that I know that hasattr doesn't do what I thought it does or what
> the name implies it does, it has little or no utility for me. In the
> future, I'll just write a try...except block and catch errors if the
> attribute doesn't exist.

The try/except block *also* requires you to break your train of
thought. And most of the time the error case just isn't important. You
sound like you are over-engineering it and focusing too  much on
performance instead of on getting things done. Like those people who
learn that it saves an usec to copy a built-in function into a defalt
arg (def foo(arg1, len=len):  ...) and then overuse the trick even
when the time it took them to write the exra line is more than the
time they'll save in a lifetime in execution time.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

> On 2010-08-23, at 10:56 AM, Guido van Rossum wrote:
> On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson  
> wrote:
>> 2010/8/23 Yury Selivanov :
>>> 1) I propose to change 'hasattr' behaviour in Python 3, making it to 
>>> swallow only AttributeError exceptions (exactly like 'getattr').  Probably, 
>>> Python 3.2 release is our last chance.
>> 
>> I would be in support of that.
> 
> I am cautiously in favor. The existing behavior is definitely a
> mistake and a trap. But it has been depended on for almost 20 years
> now.
> 
> I recommend that you create a patch, apply it, run the *entire* stdlib
> test suite and see how much breaks. That will give you an idea of the
> damage to expect for 3rd party code.

Have done a little testing.

The patch to builtin 'hasattr' function is trivial, and can be found 
attached to this letter (with the corresponding unittest):

-if (!PyErr_ExceptionMatches(PyExc_Exception))
+if (!PyErr_ExceptionMatches(PyExc_AttributeError))

So, after applying it, hasattr swallows only AttributeError exceptions.


All tests from Python 3.2 test suite were good.

After that, I've applied the patch on Python 2.6 and tested SqlAlchemy,
Django and Twisted.

- SqlAlchemy has failed three unittests, but those tests were designed
specifically to handle 'hasattr' weird behaviour, so we can consider
the change has no impact on SqlAlchemy.

- Twisted - failed 3 tests out of ~3200, but it fails them on the same
machine on an unpatched Python too, and they seem unrelated.

- Django - all tests passed.


I tested our internal company framework (~100K LOC) and, again, all
tests passed.




hasattr.patch
Description: Binary data
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Guido van Rossum]

Yuri, I think you are making a good case (though I would like for you
to be a good citizen and use the bug tracker to submit this for
review). Benjamin, what do you think?

--Guido

On Mon, Aug 23, 2010 at 7:14 PM, Yury Selivanov  wrote:
>> On 2010-08-23, at 10:56 AM, Guido van Rossum wrote:
>> On Mon, Aug 23, 2010 at 7:46 AM, Benjamin Peterson  
>> wrote:
>>> 2010/8/23 Yury Selivanov :
 1) I propose to change 'hasattr' behaviour in Python 3, making it to 
 swallow only AttributeError exceptions (exactly like 'getattr').  
 Probably, Python 3.2 release is our last chance.
>>>
>>> I would be in support of that.
>>
>> I am cautiously in favor. The existing behavior is definitely a
>> mistake and a trap. But it has been depended on for almost 20 years
>> now.
>>
>> I recommend that you create a patch, apply it, run the *entire* stdlib
>> test suite and see how much breaks. That will give you an idea of the
>> damage to expect for 3rd party code.
>
> Have done a little testing.
>
> The patch to builtin 'hasattr' function is trivial, and can be found
> attached to this letter (with the corresponding unittest):
>
> -        if (!PyErr_ExceptionMatches(PyExc_Exception))
> +        if (!PyErr_ExceptionMatches(PyExc_AttributeError))
>
> So, after applying it, hasattr swallows only AttributeError exceptions.
>
>
> All tests from Python 3.2 test suite were good.
>
> After that, I've applied the patch on Python 2.6 and tested SqlAlchemy,
> Django and Twisted.
>
> - SqlAlchemy has failed three unittests, but those tests were designed
> specifically to handle 'hasattr' weird behaviour, so we can consider
> the change has no impact on SqlAlchemy.
>
> - Twisted - failed 3 tests out of ~3200, but it fails them on the same
> machine on an unpatched Python too, and they seem unrelated.
>
> - Django - all tests passed.
>
>
> I tested our internal company framework (~100K LOC) and, again, all
> tests passed.
>
>
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Yury Selivanov]

> On 2010-08-23, at 10:37 PM, Guido van Rossum wrote:
> Yuri, I think you are making a good case (though I would like for you
> to be a good citizen and use the bug tracker to submit this for
> review). Benjamin, what do you think?

NP, issue #9666 ;-)

-
Yury
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 'hasattr' is broken by design

[Benjamin Peterson]

2010/8/23 Guido van Rossum :
> Yuri, I think you are making a good case (though I would like for you
> to be a good citizen and use the bug tracker to submit this for
> review). Benjamin, what do you think?

I will have a look right now.



-- 
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com