On Mar 27, 11:02 am, "Adam Atlas" <[EMAIL PROTECTED]> wrote: > On Mar 26, 4:55 pm, "Matimus" <[EMAIL PROTECTED]> wrote: > > > I think that is what the "code" module is for. Maybe not exactly what > > you were expecting, but the capability you describe is already there. > > Being able to access its own interpreter is one of the things that > > makes Python a dynamic language. > > That's not really what I'm after. Believe me, I've searched the > standard library in length and breadth. :) The `code` module runs in > the same interpreter, basically executing as though it were just a > separate module (except interactively). What I'm interested in is > accessing Python's C API for CREATING interpreters -- i.e. whatmod_pythonor > anything else embedding the Python interpreter would do. > Creating a whole environment, not just a module; it has its own `sys` > parameters, its own `__builtin__`, and so on, so you can safely mess > with those without changing them in the parent interpreter. > > So far, I've tried ctypes, and it doesn't work; then I noticed the > docs said it wouldn't anyway. But I think I'll try writing a C > extension module. This could be interesting for sandboxing &c.
Having played around in that area more than most I would say it would be entirely possible to create a C extension module for Python that would allow one to create additional sub interpreters. The question only becomes how does one want to make use of it and make the interface look like. A simple interface may be to direct that in named sub interpreter import this module. If named sub interpreter didn't exist then it would be created. To have code run in the separate interpreter on an ongoing basis, the imported module could create a thread. Personally though I don't like triggering side affects from module import, so might be better to say import this module and if that is successful then call named object in that module with specified arguments. The latter is better in that it allows data to be passed across as well, although you would want to limit to basic data types for various reasons. The only tricky issues in this are ensuring that thread state is given up when calling out of the first interpreter and ensuring that new thread state is created against the new interpreter before calling into it. You can't use the same thread state as then you get into issues with exceptions about running in restricted mode. Another issue to contend with is whether you allow sub interpreters to be deleted. This can be tricky for various reasons. First is that when deleting sub interpreters anything registered with atexit.register() isn't run as that only happens for main Python interpreter, but one can ensure they are by calling sys.exitfunc() within that sub interpreter. Even then, if you have separate threads running in that sub interpreter and they don't register an atexit function to allow themselves to be killed, it could hang the attempt to delete the sub interpreter. Even if that works, you may find that some third party extension modules can't cope with sub interpreters being killed through caching of the interpreter reference, something that might end up point elsewhere after the sub interpreter is killed. There are also other little minor details that one has to worry about like providing a sys.argv as sub interpreters don't have that but some Python modules blindly expect to be able to access it even though they aren't the actual main function of a program. Also possibly need to restrict access to sys.stdin from sub interpreters and only allow main interpreter to use it etc etc. Anyway, what you want to do is something I have thought about before in relation to mod_wsgi and whether there is any use for the ability to call between sub interpreters. In the context of mod_wsgi though such a thing is probably just an avenue for creating more mischief, especially within a hosting environment where different peoples applications may be running in different sub interpreters. On the other hand, it might be useful in a standalone Python based WSGI web server which you have more direct control over. It might take a bit of design work as to how to do it in practice, but you could create different sub interpreters through the module for distinct WSGI applications. The main interpreter could be running the web server and somehow then hand off a WSGI environ etc off to a manager module in the other sub interpreter that then deals with communicating with the WSGI application in that sub interpreter for that specific request. You could conceivably have a hand off arrangement whereby when an application code base changes that you start routing requests for URL subset to new instance of application in new sub interpreter and kill off old sub interpreter when able to. Would certainly be an interesting area to look at. For mod_wsgi at least though I ruled out allowing sub interpreters to be killed off and allowing reloading of a complete application by such a handoff mechanism as too dangerous a feature in an ISP based web hosting environment as user code could too easily hang the Apache process when trying to kill off the sub interpreter. Anyway, you have got my curiosity going again as to how hard it would be to write such a module. Personally I don't think it would be that hard at all and maybe when I get first version of mod_wsgi out I might have a play with the idea. :-) Graham -- http://mail.python.org/mailman/listinfo/python-list
