I decided to try using the ast module to see how difficult or not it was to use for metaprogramming. So I tried writing a decorator that would perform a simple transformation of a function's code. It was certainly not as easy as I had guessed, but I did succeed so it's not impossible. The issues I encountered might suggest changes to the ast module, but since this is my first time messing with it I think it's equally likely I'm just ignorant of the best way to handle them. Questions:
-If I have the source to a single function definition and I pass it to ast.parse, I get back an ast.Module. Why not an ast.FunctionDef? -An ast.Name always has an ast.Load or ast.Store "context" associated with it. This is a bit odd, since based on the actual context (what the parent of the ast.Name node is) it is always clear whether the variable needs to be loaded or stored -- loaded when the node is the child of an expression and stored when the node is the target of an assignment. Is there some weird corner case that necessitates this redundancy? It adds a lot of noise when you're trying to interpret trees, and the ast.Load and ast.Store objects themselves don't seem to contain any data. -Why can't orelse for ast.If and ast.While default to empty []? If you make an ast.While object by hand but don't specify that orelse is empty, you get an error when later trying to compile it. This seems silly since by default not specifying an else in code results in the ast module generating a node with orelse=[]. -Why can't keywords and args for ast.Call default to empty []? Same problem as with orelse. -Why can't I eval a functiondef to get back a function object? As it stands, I have to work around this by giving the functiondef a unique name, exec'ing the AST, and then doing a lookup in locals[] to get the resulting function object. This is a nasty hack. -The provided NodeTransformer class is useful, but provides no way (that I can see) to replace a single statement with multiple statements, because multiple statements would constitute multiple nodes. To work around this, I take the code I want to swap in, and wrap it in a "while 1:" block with a break at the end to ensure it only runs once and doesn't loop. Again, a kludge. -The module provides no means to convert an AST back into source code, which would be nice for debugging. -It would be nice if decorators were passed a function's AST instead of a function object. As it is I have to use inspect.getsource to retrieve the source for the function in question, and then use ast.parse, which is a bit inefficient because the cpython parser has to already have done this once before. Again, it feels like a hack. Making decorators take AST's would obviously be a compatibility breaking change, since you'd then have to compile them before returning, so alternatively you could have "ast decorators" that would use a different prefix symbol in place of @, or you could change it so that decorators got passed the AST but the AST had a __call__ method that would cause the AST to parse itself and become the resulting function object in place and then execute itself, so until the first time it was called it was an AST but after that was a function object (it would 'lazily' become a function). I think that wouldn't break most code. I have some ideas for what an easier API would look like but I want to be sure these are real issues first and not just me doing it wrong ;) Regards, Joe -- http://mail.python.org/mailman/listinfo/python-list
