Thanks everyone for the comments. I had previously thought about the possibility of the classes keeping track of their instances. I guess this could probably be done quite transparently with a decorator too (as we have many different types of objects being collected together). The only issue is that this approach forces you to use what are essentially global variables, whereas the searching through the stack method allows you to use the structure of the program to organise what objects each 'magic' function sees. Is this a good idea or not? I'm not entirely sure. I think that personally I would lean towards using this method of classes keeping track of their instances. It's not entirely my decision so I'll see what the others say about it.
Any comments on this possibility: classes could keep track of their instances, and also keep track of which function or module the instances were defined in. Then, the magic functions could pick out objects defined in the same function or module rather than looking at the stack. This would achieve a similar thing, but is there any great advantage in doing it this way? My first thought is that you'd still have to go digging around in the stack to do this, but just not as much. Also, does anyone know of any specific things I should be aware of in taking this stack searching approach? I'm thinking of, for example, any planned changes in the execution model of Python or the inspect.stack() function in the next version of Python. Paul, "Your users are *scientists*, and you don't trust their intellectual ability to learn a programming language as simple as Python?" Well, it's not quite as simple as that. One thing is that we're not going to be able to force people to use our package. We believe it's going to be considerably better - particularly in terms of ease of use and extensibility - than the existing alternatives, but one of the factors that will affect how many people start using it is how simple we can make it to do basic things that they'll be familiar with. Many scientists are using Python now, but it's not yet quite well known enough that we can just assume that people will know it, and having to learn the details of a new programming language is a considerable disincentive for someone thinking about switching to a new piece of software (even if, as you say, Python is not the most difficult language to learn). Although the difference between the two pieces of hypothetical code I presented seems quite trivial to an experienced programmer, I think that the clarity and simplicity of the version that uses the magic functions might make a difference. The difference between being able to define and run a model with 10 lines or 20-30 lines of code might, somewhat perversely, be a significant factor. (The example I gave was simplified to illustrate what was going on, but the actual situation is more like you have 5 or 6 different types of object, each of which uses other types of object to initialise themselves, so that the magic function approach really reduces the length of the program considerably.) So, there's an aspect of PR about our wanting to have something like the magic functions, but it's not entirely about self promotion, because we think that in the long term it will be better for the users if they switch to using our package (or something like it). The reason being that the alternatives available at the moment all use their own custom made programming languages which have nothing like the power of a well developed general purpose language like Python, and are much more difficult to use and extend. One of them is a stack based language of all things! Carl, "Even if you implement magic functions, don't get rid of the straightforward "hard way"." Absolutely not! A very good point. In fact, the magic functions don't actually do any work themselves, they just create and call the 'hard way' functions (which are still visible to the user). They're an additional layer of abstraction which you can choose to use or not use. And actually, there will be situations where there is no alternative but to use the 'hard way'. We already learnt this lesson: a couple of our magic functions were behaving differently and causing some odd behaviour, so we changed them and now we're working on building a more consistent and explicit interface (and enforcing it works as expected with the unit testing module, a tedious but hopefully very useful exercise in the long run). -- http://mail.python.org/mailman/listinfo/python-list
