Re: Should I use "if" or "try" (as a matter of speed)?
It really does depend. For instance, some other programmers where I work came up with a way to represent a hierarchical, somewhat random data set by creating each object and then adding attributes to those for each subobject, and so on down the tree. However, you could never really be sure that the object you wanted was really there, so for every access call they just wrapped it in a try ...except loop. Now that may seem like a good way to go, but when I rewrote some code to use hasattr() instead, it ran a lot faster. So yeah, exceptions can be handy, but if you code requires exception handling for everything, you may want to rethink things. Steve Juranich wrote: > I know that this topic has the potential for blowing up in my face, > but I can't help asking. I've been using Python since 1.5.1, so I'm > not what you'd call a "n00b". I dutifully evangelize on the goodness > of Python whenever I talk with fellow developers, but I always hit a > snag when it comes to discussing the finer points of the execution > model (specifically, exceptions). > > Without fail, when I start talking with some of the "old-timers" > (people who have written code in ADA or Fortran), I hear the same > arguments that using "if" is "better" than using "try". I think that > the argument goes something like, "When you set up a 'try' block, you > have to set up a lot of extra machinery than is necessary just > executing a simple conditional." > > I was wondering how true this holds for Python, where exceptions are > such an integral part of the execution model. It seems to me, that if > I'm executing a loop over a bunch of items, and I expect some > condition to hold for a majority of the cases, then a "try" block > would be in order, since I could eliminate a bunch of potentially > costly comparisons for each item. But in cases where I'm only trying > a single getattr (for example), using "if" might be a cheaper way to > go. > > What do I mean by "cheaper"? I'm basically talking about the number > of instructions that are necessary to set up and execute a try block > as opposed to an if block. > > Could you please tell me if I'm even remotely close to understanding > this correctly? > -- > Steve Juranich > Tucson, AZ > USA -- http://mail.python.org/mailman/listinfo/python-list
Re: OO design
I also have a little trouble with creating megaclasses. Usually I just
try to think about what things are a little bit, and how I'm going to
be using them. I think somebody else suggested a top down approach,
and that makes a certain amount of sense.
But at this point, you're probably getting tired of all the vague
advice. It sounds like you don't have a very complicated set of
concepts to deal with, so you probably won't wind up with too many
classes, but having a single giant one is probably inconvenient.
Here's how I would do things (although keep in mind, I only have a
vague understanding of what you really need to do..)
First off, you have your (x,y) datasets. It sounds like you perform
most of your operations on those, so they'd probably be a good idea for
a class.
You may also want a class for the (x,y) points themselves, especially
if you have any operations that collapse a point set to a single value
or values (like your means and peaks and stuff.)
So the methods on your (x,y) dataset probably want to be restricted to
your mathematical operations. Although you'll also definitely want to
look at the representation that gets passed onto your other classes.
Thinking beyond those basic classes, you probably want to now think
about your input and output operations. A general way of doing OOP for
things like this is to try to abstract your input and output operations
into objects based on what kinds of inputs or outputs they are. In
this case it sounds like you have files, which are both input and
output, and plots, which are just output. Both seem to take either
single x,y sets or a number of sets. So each I/O object would appear
to be some sort of object container for your x,y sets.
Take the plot object. It will probably be fairly simple, just
something you pass sets of x,y lists to and it plots them. You may
want some more capabilities, like naming the sets, or giving the axes
units, or whatever, but you can probably add that stuff later.
Now for the file object. Luckily, python already includes a file
object, so you already have something to base yours on. For the files,
you have two basic operations, the read and the write. For the read,
you'll need to pass in a filename, and have a way to return the
datasets (like iterating over a data file produces a dataset, hm...).
For output, you'll again need a filename, and a set of datasets to
write. For both operations, you may also want to provide formatting
options, unless you can figure it out from the filename or something.
So the ultimate hierarchy might look something like this:
class point:
self.x
self.y
class dataset:
self.data = [ #list of points ]
self.units = '' #?? maybe
def sum
def mean
# and so on
class datafile:
self.filename
self.filetype
self.datasets = [ #list of data sets? ]
def read
def write
## and so on
class plot:
self.datasets = { #dictionary of data sets, each named?}
self.xaxis = ''
self.yaxis = ''
def makeplot
## and so on
Looking at it like that, it might make some sense to come up with
something like a dataset collection class that you can pass to plot or
datafile, but if you can use something like a list (if the data is
fairly arbitrary) or a dictionary (if the datasets have names or some
other way of signifying what they're about), that will probably be
enough. Anyway, that's just a rough sketch, hopefully it will give
you some ideas if it doesn't seem like the best solution.
--
http://mail.python.org/mailman/listinfo/python-list
pyUI?
I think someone built a gui on top of pygame a while back, I think it was called pyUI or something, let's see what google gives me here, ah, here we go http://pyui.sourceforge.net/ Hope that helps. Looks like neat stuff anyway... -- http://mail.python.org/mailman/listinfo/python-list
Re: Rich Graphics?
for widgets, try pyUI http://pyui.sourceforge.net/ hope that helps, looks cool and all.. -- http://mail.python.org/mailman/listinfo/python-list
build flow? SCons? AAP? process creation?
Hey guys, here's a little question. I'm looking for something like a build system, where I can set up a bunch of jobs with dependencies, and have them run in parallel. Ideally, I'd like a system where jobs can be run in parallel, all the stdout and stderr for each job is kept in a database of some kind, jobs can be anything that can be run at a command line, and a failing job won't cause the whole run to fail. I've been looking into some of the standard stuff (GNU make) and some of the more pythonic stuff like SCons and AAP. SCons seems to do parallel execution, which is nice, but I can't figure out from the docs if it's possible to keep stdout and stderr somewhere. AAP seems to have a nice interface, but it doesn't do parallel. So if I do wind up having to write this thing myself, I've been checking the docs on process creation, and have a couple questions if anyone can fill me in. It looks like the os.spawn* commands can start nonblocking sub-processes, but there doesn't seem to be a way to get stdout and stderr. On the other hand, the popen commands make it easy to trap stdout and stderr, but I guess I'd have to do the thread setup and spawning myself. Is there another alternative that I'm missing here? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: build flow? SCons? AAP? process creation?
good point, I hadn't checked the docs too closely. Shame it's only in 2.4 though, we're still running 2.2 around here, but it would be nice to have a reason to upgrade, anyway. Thanks for the pointer! -- http://mail.python.org/mailman/listinfo/python-list
python concurrency proposal
Alright, so I've been following some of the arguments about enhancing
parallelism in python, and I've kind of been struck by how hard things
still are. It seems like what we really need is a more pythonic
approach. One thing I've been seeing suggested a lot lately is that
running jobs in separate processes, to make it easy to use the latest
multiprocessor machines. Makes a lot of sense to me, those processors
are going to be more and more popular as time goes on. But it would
also be nice if it could also turn into a way to make standard
threading a little easier and trouble free. But I'm not seeing an easy
way to make it possible with the current constraints of the language,
so it seems like we're going to need some kind of language improvement.
Thinking of it from that perspective, I started thinking about how it
would be easy to deal with in a more idealized sense. It would be nice
to abstract out the concept of running something in parallel to
something that can be easily customized, is flexible enough to use in a
variety of concepts, and is resonably hard to screw up and fairly easy
to use. Perhaps a new built-in type might be just the trick. Consider
a new suite:
pardef (self, , arguments...):
self.send(, , arguments)
self.receive(, arguments)
return arguments
yield arguments
so the object would then be something you can create an instance of,
and set up like a normal object, and it would have other interface
functions as well. Consider your basic vector add operation:
import concurrent
import array
pardef vecadd(self, concurrent.subprocess, veca, vecb, arrtype):
import array
output = array.array(arrtype)
for a,b in zip(veca, vecb):
output.append( a + b)
return output
a = array.array('d')
b = array.array('d')
for i in range(1000):
a.append(float(i))
b.append(float(i))
h1 = vecadd(a[:500], b[:500], 'd')
h2 = vecadd()
h2.veca = a[500:]
h2.vecb = b[500:]
h2.arrtype = 'd'
h1.run()
h2.run()
c = h1.result + h2.result
You can see a few things in this example. First off, you'll notice
that vecadd has the import for array inside it. One of the most
important things about the pardef is that it must not inherit anything
from the global scope, all variable passing must occur through either
the arguments or .receive statements. You'll also notice that it's
possible to set the arguments like instance values. This isn't as
important in this case, but it could be very useful for setting
arguments for other pardefs. Take this example of your basic SIMD-ish
diffusion simulation:
import concurrent
pardef vecadd(self, concurrent.subprocess, right, left, up, down,
initval):
current = initval
maxruns = 100
updef = not (isinstance(up, int) or isintance(up, float))
downdef = not (isinstance(down, int) or isintance(down, float))
rightdef = not (isinstance(right, int) or isintance(right, float))
leftdef = not (isinstance(left, int) or isintance(left, float))
for i in range(maxruns):
if updef:
upval = self.receive(up, 'up')
else:
upval = up
if downdef:
downval = self.receive(down, 'down')
else:
downval = down
if rightdef:
rightval = self.receive(right, 'right')
else:
rightval = right
if leftdef:
leftval = self.receive(left, 'left')
else:
leftval = left
current = (upval + downval + leftval + rightval) / 4
if updef:
up.send('down', current)
if downdef:
down.send('up', current)
if rightdef:
right.send('left', current)
if leftdef:
left.send('right', current)
return current
diffgrid = {}
for x, y in zip(range(10), range(10)):
diffgrid[(x, y)] = vecadd()
for x, y in zip(range(10), range(10)):
gridpt = diffgrid[(x, y)]
gridpt.initval = 50.0
if x == 0:
gridpt.left = 75.0
else:
gridpt.left = diffgrid[(x-1, y)]
if x == 10:
gridpt.right = 50.0
else:
gridpt.right = diffgrid[(x+1, y)]
if y == 0:
gridpt.down = 0.0
else:
gridpt.down = diffgrid[(x, y-1)]
if y == 10:
gridpt.up = 100.0
else:
gridpt.up = diffgrid[(x, y+1)]
for coord in diffgrid:
diffgrid[coord].run()
for x, y in zip(range(10), range(10)):
print '(%i, %i) = %f' % (x, y, diffgrid[(x,y)].return())
Now sure, this example is a little contrived, but it shows the utility
of allowing
Re: python concurrency proposal
Hey, some responses, let's see... Peter Tillotson wrote: > I'd really like to see a concurrency system come into python based on > theories such as Communicating Sequential Processes (CSP) or its > derivatives lambda or pi calculus. These provide an analytic framework > for developing multi thread / process apps. CSP like concurrency is one > of the hidden gems in the Java Tiger release (java.util.concurrency). > The advantages of the analytic framework is that they minimise livelock, > deadlock and facilitate debugging. Yes, a CSP-like system would be kind of nice. Of course, to really pull it off, you'd probably need some kind of primitive to represent a simple process. Which is kind of what I'm proposing. It's kind of a primitive version, but an object to easily represent processes and communication channels would be a big help, I'd imagine. Once a primitive is in place, I believe it would be fairly easy to build up a full set of CSP-ish primitives to help assemble full systems. > I'm no expert on the theory but i've developed under these frameworks > and found them a more reliable way of developing distributed agent systems. > > You may also be interested in looking at > http://sourceforge.net/projects/pympi Ah yes, pympi, that's good stuff. It does suggest that I might need to add blocking and non-blocking version of send and receive, there might be a need for that. Especially the non-blocking receive, that looks very handy. And maybe a .status for polling the status of running jobs. It does go a lot further than I do with this proposal, it adds all that stuff for collecting the jobs, running them as a group, gathering results, and so on. This proposal could probably use an extra library to provide a bunch of those type of operations, but I wanted to start a little bit small here with just the proposal of a process primitive. That seems like it would be a good first step. Of course, what I'm really hoping for is that at some point you could do something like this: import mpi pardef vecadd(self, mpi.mpiproc, ...) and so on. I'm not really all that concerned about the communication channels and process distributions work, I suspect that many people will want to try different methods like MPI, or the standard pthread wrapper with some kind of standard queue communications channel, or pyro, maybe even Kamaelia. I'm just proposing the primitive for it. So, if there's anything else you'd like to see it work like, be sure to let me know. Thanks for the input! - Corey -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
> Yes. Parallelism certainly deserves attention, and I believe > "amateurs" are likely to help in the breakthroughs to come. I > further suspect, though, that they'll be amateurs who benefit > from knowledge of existing research into the range of documented > concurrency concepts, including CSPs, tasks, guarded methods, > microthreads, weightless threads, chords, co-routines, and so on. Yes, there are lots of different concepts, even in python, there's pympi (as was mentioned), the standard python thread library, the subprocess library, generators, microthreads and stackless, not to mention Candygram, PyLinda, ATOM, Kamaelia (get to that in a minute), and other things you can search for on the web. My motivation here is just to see if I can find some lowest common denominator, to try to simplify this stuff to the point where the whole concept is a little easier to use, and the plumbing can be hidden away somewhere so "amateurs" don't have to worry about it (too much) if they don't want to. Now to be more specific, there does seem to be a lot of work with generators to set up concurrency, and that's fine, but it does seem like it takes a bunch of scaffolding and a different way of looking at things, and it's not really obvious to me how it can scale up on multiple processor systems with the GIL still in place. Now I'm not sure that this will be the answer to all the problems, but breaking the global address space and making it easy to break up jobs into small communicating chunks seems like it would be a good way to go. Or maybe I'm missing something? Is there anything you'd care to elaborate on? -- http://mail.python.org/mailman/listinfo/python-list
Re: python concurrency proposal
OK, thanks for all this criticism, you've obviously taken some time
here, guess I'll see if I can help clear some of this up
Michael wrote:
>
>
> On the surface of it, what you've described resembles Kamaelia[1] -
> specifically in the way used in the Axon Shell [2]. In other ways it
> differs wildy. I think it's interesting because this (or some of this)
> could be used as syntactic sugar for Kamaelia.
> [1] http://kamaelia.sourceforge.net/Home
> [2] http://kamaelia.sourceforge.net/AxonShell.html
>
> The similarity is this: a pardef appears to be a function that can run,
> and be made to run in parallel with other functions. In Kamaelia we use
> generators to achieve precisely this.
Hey, you got it, great!
> However, given pythonic is a loaded term (beauty is in the eye of the
> beholder), I'd personally say that there's some elements of your syntax
> that suprise me, especially given the (IMO valid) approach of not being
> able to access inside the pardef.
>
> Since there are some strong similarities between what you've written and
> Kamaelia it seems sensible to discuss them.
>
> If you have your pardef:
> pardef (self, , arguments...):
> self.send(, , arguments)
> self.receive(, arguments)
> return arguments
> yield arguments
>
> This maps to this: (in your terms :) )
>
> class ():
> Inboxes = [ , , ] # Explicit is better than implicit
> Outboxes = [ , , ] # There are defaults available...
> def __init__(self, arguments ...):
> // standard initialisation //
> super(, self).__init__()
>
> def main(self):
> // local
> // Initialisation
> while 1:
> do stuff
> yield //some value// (relinquish control to scheduler)
> return
> // We don't have the concept of a result, though this
>would be useful, though we do have something
>similar so this would be doable //
>
> Inboxes and Outboxes are used as declarations to allow the baseclass
> (component) to initialise some datastructures for communications. These
> declarations are actually any iterable, and these days we tend to use
> dictionaries because it simplifies documentation.
Yes, it would map in a more detailed level to a class of some kind with
the methods indicated. Although, I'm imagining that for different base classes, the init and main loops would be implemented in
different ways. Different types of communication channels (shared
memory copies, file based pickling, socket communication, and so on)
would also have different implementations as well, beyond Inboxes and
Outboxes. This could make for some much more complicated base classes.
It also suggests some problems, notably that of coercing different
messages from different types of base classes (i.e., suppose the
standard library os and system classes get implemented as threads, but
you still want to send and receive messages to and from them from
processes and clustered processes).
>
> An example here might control the behaviour of a sprite onscreen. So,
> suppose we have a sprite:
>
>
> Unlike your system, our components (your pardefs), don't know
> about each other and only talk to inboxes AND outboxes - these
> are connected at a higher level by something enclosing them.
>
> Graphline(
> Walker = SimpleSprite(walkerImage, 10,10),
> WalkerLogic = WalkInASquare(10,10,400,400,50),
> linkages = {
>("WalkerLogic", "position") : ("Walker", "position"),
>("WalkerLogic", "orientation") : ("Walker", "rotation"),
> }
> )
>
> This graphline can then be activated or run like any other component.
Yes, I've become a little worried about the casual way I'm pulling the
pardefs together. I did want to keep the definition fairly primitive,
and then try to pull the jobs together using standard python objects
like lists and dictionaries and such, but given that pympi and Kamaelia
use more custom methods, it may be something to think about. It might
simplify the blocking/non-blocking behavior, which I'll get to soon...
> [[ For more complete examples, grab Kamaelia from the website,
>download and run :) ]]
>
> You'll note from the above that clearly there are aspects to it where
> your ideas could be used as syntactic sugar. Some of the benefits (like
> collapsing __init__ and main together) can be gained by using decorators
> in python 2.4.
I've actually been thinking that decorators may be the way to go to set
the base class of the pardef, either that or have it be another class
modifier. Neither seems very ideal, but having it after self also seems
kludgy.
> You'll also note there are some areas where your ideas increase parallel
> safety (such as when sending mutable data structures), at the expense on
> increasing the cost of communication.
Yes, the hope is that the ability to have different base classes that
implement the same interface will allow peop
Re: python concurrency proposal
Mike Meyer wrote: > [Rest elided] > > This really has a lot in common with SCOOP. SCOOP makes concurrent > stuff easier, but i'm not sure it fits well with Python. I'll describe > it, on the off chance you may get some ideas from it. See http://archive.eiffel.com/doc/manuals/technology/concurrency/ > for > full details. Eiffel! I should have thought of that. I may have to dig all my old Meyer books out of storage.. > > Like pardef, you use SCOOP by creating objects that are going to run > concurrently. Unlike pardef, the objects are - well, objects, not just > restricted to functions. This has deep implications. For instance, one > of the things that sets SCOOP apart from other concurrency systems > that I know if is that the program doesn't have code to create > "processors" (threads, process, whatever - places for the code to > run). From my reading of your description, this isn't the case for > pardef. Invoking a pardef causes a new processor to be created, and > when the pardef returns, the processor is finished. Well, not quite, the pardef is actually an object, and you can certainly put all kinds of objects inside it if you want to. The pardef object is really more of an interface specification, where the actual implementation of most of the methods is really defined in the base class. The body of the pardef really just defines what happens during a .run() call. The whole 'def' in pardef and the use of returns (and yields) may be kind of misleading, the return (and yield) are really just ways to send info back to the calling scope without having to come up with some wierd way of referencing it. What I'm envisioning can return a single value, or stay resident and return multiple values (through yield or .send calls) or whatever. > > Like pardef, SCOOP separates the specification of the kind of > processor from the rest of the code. It carries things further than > pardef though. You specify a set of processors at build time, and the > support system takes care of getting objects on the right processor. > > Synchronization and protection is all handled via method > invocation. There are no new primitives for locking objects, explicit > synchronization, or communications. Yeah, I've been trying to think of some way to setup channels in a better way. Tagged channels can certainly be used, but method (or method-like) invocations would be more elegant in more than a few situations. On the other hand, I've also been trying to avoid locking people into a fully OO style in order to use concurrency, it would be nice to allow people who don't want to define everything as an object to use parallelism. But it can get ugly that way. Maybe something like a "property" declaration to optionally simplify things, hmm > > Calls to methods of a separate object are non-blocking. The calls are > queued, and executed later. Reading an attribute from a separate > object is a blocking action. Calling a method with a separate object > as an argument causes the call to block until the separate arguments > are all locked. There are rules - enforceable by the compiler - about > what you can do with separate objects that prevent a lot of the things > that cause headaches when doing concurrent programming. Undoubtedly enforced by contracts, Eiffel is nice that way. Some things I do miss from stricter languages. But then again, having to define everything as an object does drive me nuts. > > So instead of "run", you instantiate a separate object. Instead of > ".send", you just invoke a method on the seperate object, and the > arguments get sent to it. Instead of ".receive", you read an attribute > from a separate object. Instead of ."kill", you let the object be > garbage collected (or maybe those aren't the same thing). Return is > missing, because it's an object, not a function. On the other hand, > you could define an attribute that's the "return value", and reading > it fetches that value. > > The problems with Python is that "separate" is really a property of a > variable, not an object. To see the difference, if I declare an object > separate, and it creates an attribute, and I read that attribute, then > that object is separate for me. But in the object that created it, it > isn't. This is one major clash with Python. The other is the > critical distinction that SCOOP places on reading vs. calling a > feature. What does: > > a = Seperate_Object() > x = a.thing > x() > > do with SCOOP semantics? > > Oh well. I think SCOOP does the job you want of "making things > easier", and fits well in an OO language. Maybe you can find good > ideas in it. > > http://mail.python.org/mailman/listinfo/python-list
