Carl Banks wrote:
On Jul 29, 6:42 pm, Matthew Fitzgibbons <[EMAIL PROTECTED]> wrote:
Carl Banks wrote:
Much like in Steven D'Aprano's example, still the only actual code
snippet I've seen, it seems that this can easily be done with a simple
explicit test by having all no-advance filters return None and testing
with "if x is not None". So it doesn't pass my criterion of being not
replaceable with simple explicit test.
Maybe that's not workable for some reason. Perhaps if you'd post a
code example that shows this, rather than just talking about it, you
might be more persuasive.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
The no-advance filters have to return the object because I don't just
forget about it; I evaluate whether I pass it to the next filter or drop
it in a completely different queue for use in the next stage of the
operation. True means 'I'm ready to move on to the next stage,' False
means 'Do the filter thing some more.'
I think I see what you're saying, and yeah I guess that could really
take advantage of polymorphism between very different types.
Furthermore, the argument that I should just change my API to make a
'simple test' work is not very convincing.
I wasn't suggesting you change it: I was trying to ascertain whether
it would have suffered much if you had written it with explicit tests
in the first place, or if Python didn't even have magical booleans.
Yes it would have suffered. I chose the implementation I did because it
made the most sense to me and was the most flexible (a key requirement).
Instead of cobbling together my own way, I used the way that Python gave me.
Python doesn't have magic booleans. They are instead a very well-defined
language mechanism that isn't perfect for every circumstance, but is
pretty good for the most part. I wanted to do meaningful boolean tests
on various objects, so I used the mechanism that my language gave me.
The natural, obvious way for
a filter to work is to pass through the data it operates on; why on
Earth would it return None? I want to DO something with the data. In
this case, make a decision about where to pass the data next.
If you don't mind me asking: what do you do actually DO with a zero or
empty list?
Depends on exactly what the next stage is. Typically, zeros and empty
lists are not meaningful for the next stage, so they get dropped then if
they make it through. I don't want to restrict what gets passed through,
though, because I could end up with several meaningful data types,
making a simple test again impossible. So I pass everything through and
let the next stage decide.
In Java,
to accomplish this I would have to do lots of introspection and value
checking (adding more any time I came up with a new kind of input), or
make a new kind of interface that gives me a method so I can do a
'simple test' (including wrappers for ints and arrays and anything else
I decide to pass in down the road). But Python supports duck typing and
gives me a handy __nonzero__ method; I can rebind __nonzero__ in my
filters for my own classes, and ints and lists are handled how I want
them to be by default. So why jump through hoops instead of just using
'if x'?
Ah, so it's just happens to work. Still, just happening to work
works. (shrug)
Nonono. The point you seem to be missing is that 'if x' is very well
defined. There is nothing magic or arbitrary about what it does. It
works here and elsewhere because Python (a) chooses good default
behavior (its treatment of lists and ints, etc) and (b) gives you a way,
__nonzero__, to change the behavior to suit your needs.
I don't have any postable code (it's in a half way state and I haven't
touched it for a while), but I'll see if I can't find the time to bang
something up to give you the gist.
I wouldn't bother at this point. I was looking to see if someone
could come up with something to disprove my belief on the polymorphic
uselessness of "if x" relative to explicit tests, and if (as I
anticipated) they did not, I could claim that "if x" is really just a
glorified keystroke saver. But I now realize that the failure of this
bunch doesn't prove anything. I don't think most people even realize
why answering the question I asked would demonstrate the usefulness of
"if x".
Of course you don't _need_ to have 'if x'; after all, <sarcasm>REAL
programmers code in machine language</sarcasm>. The whole point of a
high-level language is to make life easier. Python is very dynamic and
allows duck typing so that you can use these tools to do clever things
that would otherwise be very difficult. It even gives you a handy
polymorphic mechanism to do boolean tests, which my example illustrates.
You asked for a use case for a polymorphic 'if x' that can't be easily
replaced by a simple test. I gave one. Then you asked for a code sample,
but now have dismissed my sample as not compelling without having seen
it. But here it is anyway (attached). It's pretty rough since I just
threw it together. I'm sure there are mistakes. I also dropped the DAG,
where all the queueing and decision making is handled in the actual
program. The filters and data are arbitrary but should illustrate the
setup. You care about line 66. Again, the same thing could be
accomplished in various ways; but keep in mind that data could be
_anything_, so you can't easily rewrite line 66.
Your example isn't exactly the smoking gun I was looking for, but I
guess we'll have to admit that at least one usage will suffer for not
having it.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
So you hypothesized that you can easily rewrite any 'if x' as a simple,
explicit test. I produced an example that shows this cannot be done;
therefore your hypothesis is not correct.
For most cases, you can come up with a simple test, even if it's not the
best way to implement you problem. But other times, the poylmorphic,
duck typing behavior of 'if x' allows you to make your problem much
easier, in a way a simple test can't. To deny this fact is to deny to
power of dynamic languages in general.
-Matt
"""Example where polymorphic 'if' is helpful.
Greatly simplified. A simple chain of filters. No DAG.
"""
import Queue
import random
import threading
# the data to pass to the next stage
stage_out = Queue.Queue(0)
def ordinary_nonzero(self):
"""Ordinarily, if we have a valid instance, __nonzero__ should return True."""
return True
def call_nonzero(self):
"""We want __nonzero__ to call _nonzero."""
if hasattr(self, "_nonzero"):
# _nonzero is _not_ a method!
return self._nonzero(self)
return True
class Cell(object):
"""Custom data type."""
def __init__(self):
self.length = 0
self.sets = []
self.filters = []
def __str__(self):
return "Cell:\n\tLength: %d\n\t%s\n\t%s" % (self.length, self.sets, self.filters)
class TearDownFilters(object):
"""Symbol telling us to tear down the filters."""
def __nonzero__(self):
"""We don't want these to get passed through the chain."""
return False
class Filter(object):
def __init__(self, in_queue, out_queue):
self.in_queue = in_queue
self.out_queue = out_queue
self.thread = threading.Thread(target=self._loop)
self.thread.start()
def join(self):
self.thread.join()
def _loop(self):
data = self.in_queue.get()
while not isinstance(data, TearDownFilters):
data = self._filter(data)
# mark the data so we know what filters the Cell hit
if isinstance(data, Cell):
data.filters.append(self.__class__.__name__)
# figure out what to do with the data
if data: # !!! the *magic* line !!!
# if data is True, we want to continue filtering
self.out_queue.put(data)
else:
# otherwise, put __nonzero__ back and stop filtering
if isinstance(data, Cell):
data.__class__.__nonzero__ = ordinary_nonzero
del data._nonzero
stage_out.put(data)
data = self.in_queue.get()
self.out_queue.put(TearDownFilters())
def _filter(self, data):
"""Template for Filter subclasses."""
raise Exception("Not implemented")
# if we want to do something to the object
# process the data
# rebind __nonzero__ if you want to
# return something interesting to pass on
class GenerateFilter(Filter):
"""Generates a new Cell with the given number of sets."""
def __init__(self, in_queue, out_queue, numchoices=12, setsize=3):
self.choices = range(numchoices)
self.setsize = setsize
Filter.__init__(self, in_queue, out_queue)
def _filter(self, data):
if isinstance(data, int) and data > 0:
cell = Cell()
cell.sets = []
for i in xrange(data):
set = []
for j in xrange(self.setsize):
set.append(random.choice(self.choices))
cell.sets.append(set)
data = cell
return data
class SetFilter(Filter):
"""Add a set to a Cell."""
def __init__(self, in_queue, out_queue):
self.start = 0
Filter.__init__(self, in_queue, out_queue)
def _filter(self, data):
if isinstance(data, Cell):
start = self.start
data.sets.append([start + 1, start + 2, start + 3])
self.start = (start + 1) % 10
return data
class LengthFilter(Filter):
"""Change the length of a Cell.
Stop changing the Cell if it's above a certain length.
"""
def _filter(self, data):
if isinstance(data, Cell):
# change the length
data.length = random.choice(range(10))
# create and bind the new __nonzero__ test
def new_test(self2):
if len(self2.sets) > 2 and self2.length > 3:
return False
elif self2.length > 6:
return False
else:
return True
return val
data._nonzero = new_test
data.__class__.__nonzero__ = call_nonzero
return data
class ExitFilter(Filter):
"""More interesting stuff."""
def _filter(self, data):
# etc....
# make sure all Cells get into stage_out
if isinstance(data, Cell):
def new_test(self2):
return False
data._nonzero = new_test
data.__class__.__nonzero__ = call_nonzero
return data
if __name__ == '__main__':
# make the queues connecting the filters
queues = []
for i in xrange(5):
queues.append(Queue.Queue(0))
# make the filters
sf = SetFilter(queues[0], queues[1])
gf = GenerateFilter(queues[1], queues[2])
lf = LengthFilter(queues[2], queues[3])
ef = ExitFilter(queues[3], queues[4])
# put stuff through the filters
# there aren't currently any filters that act on a list, so it gets dropped
stuff = [3, Cell(), Cell(), [4, Cell()]]
for item in stuff:
queues[0].put(item)
# teardown the filters
queues[0].put(TearDownFilters())
sf.join()
gf.join()
lf.join()
ef.join()
# check the result
print "RESULT"
while True:
try:
print stage_out.get(False)
except:
break
--
http://mail.python.org/mailman/listinfo/python-list
