Anyone who has wrapped C or C++ libraries has encountered the proxy
dilemma.
A long time ago, I naively thought that I could start by deriving my
high level python class from the c-python type, but this leads to many
difficult problems because several of the underlying methods return the
low level objects. After reading up on the subject, I learned that the
"correct" solution is to use composition rather than inheritance. It
makes sense, but it is nevertheless rather annoying.
An excellent example of this is the wxPython library, which uses
composition (proxies) and even solves the OOR (original object return)
problem that is associated with this kind of proxy oriented solution.
Conclusion: There is no really good solution to this kind of thing that
doesn't get messy. Wrapping C or C++ in python becomes difficult
whenever you have methods that return (or take as arguments) instances
of the objects that you are trying to wrap in high level python. Any
solution seems to add more overhead than is desirable (in terms of both
programmer time and run time).
This problem comes up over and over again, so perhaps it is even worth a
PEP if a change to the python language would facilitate a more
convenient solution to this nagging problem.
First, mentally set aside the idea of using composition. I understand
the set of problems that it solves, but it also creates a new set of
problems (such as function call overhead on every method call). I also
understand why composition is better than inheritance. But in order to
contemplate this proposal, it is necessary to temporarily set aside the
composition idea. This proposal involves taking another look at
something slightly more akin to the inheritance approach that we all
gave up on before.
Okay, so here is a somewhat radical proposal:
Add a writable __overload__ attribute to low level python type. This
would be assigned a python 'type' instance (or None to have no effect).
The effect is to place __overload__ at the /beginning /of the __mro__ of
the low level type, making it possible to directly alter the behavior of
the base python type without using inheritance. This means that
instances of the base type acquire the attributes of our high level
python class. It is important that the __overload__ type is checked
/before /the actual type.
Yeah, it sounds a bit scary at first. A bit too loose maybe. So the
type definition should have to explicitly enable this feature to prevent
people from doing bad things. But dwelling too much on the scariness
seems somewhat non-pythonic. It is more important that we can do really
good things than that we prevent ourselves from going out of our way to
do bad things (as long as it is not too /easy /to do bad things).
So here are some of the benefits:
1. Eliminates the duality between proxy objects and low level objects.
2. Increased speed on methods whose syntaxes are not being altered
(because no wrapper is needed).
3. Much less work than writing wrappers for every method (even if such a
task is partially automated).
Usage example:
from foolib import c_foo
class Foo(c_foo):
"""Overload of c_foo"""
def __new__(typ, *args, **kwargs):
return c_foo(*args, **kwargs)
def __init__(self, parrot_state):
c_foo.__init__(self)
self.parrot_state = parrot_state
def myfunc(self):
return 5 * self.juju(self.parrot_state) # juju method is defined
in foolib
# This makes all c_foo instances into Foo instances: Poof!
c_foo.c_footype.__overload__ = Foo
x = Foo('not dead yet') # actually creates a c_foo instance that looks
like a Foo instance
# c_foo.Add is defined in foolib and takes c_foo instances
x.Add(Foo('strange'))
x.Add(Foo('missing'))
# c_foo.GetChildren is defined in foolib and returns c_foo instances
for y in x.GetChildren():
print y.myfunc() # Wow, I am magically accessing Foo.myfunc
Yeah I know, that's an infinitely recursive inheritance; but that's
okay, we just implement the __overload__ feature such that the MRO
thingy will do the right thing for us. The semantics will be that Foo
is derived from c_foo, and all instances of c_foo behave as if the are
instances of Foo.
I used the term "radical" earlier, to describe this idea. What I meant
was that it seems to violate some basic Object Oriented principles.
However, when evaluating solutions I try not to apply any principles too
dogmatically. Theory is only relevant when it has /practical
/consequences. So I recommend not quoting theory without also defending
the relevance of the theory to the case in point. IMO, anything that
eliminates thousands of lines of boilerplate code is worth bending
theory a little.
Is it possible to get the same syntactic/semantic results without
changing python?
- Ken
--
http://mail.python.org/mailman/listinfo/python-list
