See reply BELOW in sections marked by ==========:

-----Original Message-----
From: DL Neil <pythonl...@danceswithmice.info> 
Sent: Wednesday, January 23, 2019 7:39 PM
To: Avi Gross <avigr...@verizon.net>; python-list@python.org
Subject: Re: checking protocols.

Avi

Haven't noticed an answer to this. Did I miss anything?

==========REPLY ON non-TOP as requested on 1/23/2019============
Dave,

I never saw the original message appear and thus expected no replies. I see a 
later post by Chris indicating he did not see it either. I assumed perhaps a 
moderator needed to approve it.

My question really boils down to being intrigued how the python language can be 
extended using documented protocols in more ways and having a place where you 
can see a list of the ones out there and a way to join in with your own classes 
reliably and so on.

I note there may be many unpublished protocols used internally within modules 
that also create similar structures but do not expect to be emulated.

I see interpolated comments below and will interpolate within that but it gets 
confusing.
==========END first set of remarks on 1/23/2019============
==========ORIGINAL MESSAGE BELOW if anyone wants to see in-line comments and 
supply info.===============

On 20/01/19 11:07 AM, Avi Gross wrote:
> Short question. Checking if a protocol is set up?

=do you mean that to check/require that a class exhibits a particular protocol 
we should use abstract classes - will not instantiate unless all the required 
components are coded?

========== I am not looking for any specific options but indeed including an 
abstract class might be one way to go. It might supply a reasonable way to ask 
if the class you make yourself INTENDS on running that protocol and would 
enforce the presence of the required methods somewhere in the inheritance 
chain. Of course the functions could be stubs that don't do the right thing, or 
anything passable.


> Many python improvements are changes to classes that implement a protocol.
> There are things you can do to make your own classes work with the 
> protocol by setting various dunder variables like __iter__, __next__ 
> and writing appropriate ode including throwing the right error class when 
> done.
> Similarly the "with" statement works with objects that implement 
> __enter__ and __exit__. There can be plenty of others like this and 
> more can be anticipated in the future.
> 
> So, several related questions. Tools that help a developer add 
> appropriate things to an object to implement the protocol or to test 
> if it was done right. Perhaps a function with a name like 
> is_iterable() that tells if the protocol can be applied. For the 
> specific case of an iterable, I found something that seems to work for at 
> least some cases:
> 
> from collections import Iterable
> item = [1, 2, 3, 4]
> 
> isinstance(item, Iterable)
> 
> Not sure if it would work on one I created that did the right things 
> or what it checks.

=your code should be 'approved' if it implements the next() method, etc. 
Did you try such?

==========Just to be clear, yes. That is the current state of affairs. ANY CODE 
can be run but probably fails when it does not follow the recipe. Any of many 
places that expect the iteration protocol will ask for a __iter__ and either 
catch the error or fail. If it works, they will call __next__ as often as 
needed and may or may not fail well if it is not found. And, as noted, the 
function may throw the right error object on completion or not and that may 
cause the caller to fail mysteriously or be bypassed if some other error is 
thrown instead. What I am hoping for is OPEN to multiple solutions or to be 
told there is no such need. One solution would be something that can be used to 
exercise the function and report if the protocol is being honored. In this 
case, that is fairly trivial. Just try the object in one of many places the 
iterator protocol is used. It may be less clear how you check if the 'with' 
protocol is satisfied as it may not properly close whatever it is guarding like
  closing a file or freeing a lock. If you test that you may not notice the 
file remains open and so on. But will your code also work if the user closed 
the file already on purpose?


> I am interested in a pointer to something that describes many of the 
> known protocols or extensions and maybe to modules designed sort of as 
> I said above. I am aware some protocols may be not-quite standard with 
> parts of the protocol embedded in different objects like wrappers or 
> objects returned upon a request to have a proxy and many other 
> techniques that seem to abound and allow multiple layers of 
> indirection or seemingly almost magical as in multiple inheritance 
> drop-ins and so on. That is what may make these things harder if 
> someone uses something like __getattr__ or descriptors to intercept 
> calls and provide the functionality without any actual sign of the dunder key 
> normally expected.

=Questioning similarly, I recall finding one of these - but do you think that I 
can re-find it now? Apologies.

I (too) think it would be handy to have such a list. There are many for the 
'magic methods' themselves, in all the better Py3 texts.

Yesterday I needed to add __LT__() to allow a list of class instances to be 
sorted, __EQ__ to enable a list of (other) instances to be searched (if element 
in list_of_instances), and made a class callable (__call__()). Each time I 
wondered: is this the best way to accomplish or is there already a mechanism I 
could be employing/not 'reinventing the wheel'. (perhaps incompletely!)


For your further reading pleasure (maybe):

On the this topic, one of many references is Interfaces in Python: 
Protocols and ABCs
http://masnun.rocks/2017/04/15/interfaces-in-python-protocols-and-abcs/

Which reminded me of the amusingly titled: Duck Typing vs. Goose Typing, 
Pythonic Interfaces 
https://dgkim5360.github.io/blog/python/2017/07/duck-typing-vs-goose-typing-pythonic-interfaces/

Diving into the docs (although not finding exactly what we seek):
https://docs.python.org/3/library/abc.html
- didn't include all, eg context managers:
https://docs.python.org/3/library/contextlib.html

At a deeper level:
https://docs.python.org/3/c-api/abstract.html?highlight=abstract
and
https://docs.python.org/3.6/c-api/object.html
- again, incomplete in the sense (I gained) of this enquiry.

Hopefully there's something to keep your mind occupied...

==========I appreciate the references and will follow up. Quick side note about 
the phrase "magic" is that I was reading up on Ipython and Jupiter Notebookss 
and they have yet another concept of magic based on commands starting with "%" 
as in "%run file" as they are not talking to core python directly but sort of a 
level above.

===The ideal goal is for magical features  to be invisible and magical at one 
level while being very clear and aboveboard in other ways. So, I would like to 
be able to do something that tells me clearly what some object implements as in:

from magic import protocols_detected
if ("iteration" in protocols_detected(my_object): ...

Now obviously some simple protocols that use one dunder method may be easier to 
check in a simpler way. Does the object support a sorting algorithm as in your 
example? But does that mean defining either __lt__ or __gt__ or must it be both 
as well as __eq__ and perhaps more? 

There are virus checkers that store lists of fingerprints that can be used to 
look in places within a file to guess well if it contains a known virus. I am 
thinking of an application that stores some similar fingerprint for lots of 
known protocols and when asked, produces a list like the code above, or some 
other method. Of course an object that wants to support lots of protocols using 
mix-in and abstract classes might get quite complex. And, as noted, python on a 
deeper level lets programmers be beyond devious as the simple-looking object 
turns out to have all kinds of wrappers and proxies that can obfuscate the heck 
out of figuring out what methods actually exist and in which of many related 
objects they are hidden. I honestly cannot expect any such tools to be more 
than heuristics that can get it wrong. To be concrete, you can make a class 
definition that simply does not show a definition for something like __iter__ 
but uses descriptors and decorators and other techniques that eff
 ectively create the darn thing indirectly at the time it is evaluated or when 
it is first asked for or even delay creating what is needed until needed.

The above is speculative so please don't expect me to back it up with a 
specific example. LOL!

==========Regardless, Avi.
--
Regards =dn

-- 
https://mail.python.org/mailman/listinfo/python-list

Reply via email to