On 17 Apr, 10:25, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> help progress at all. I think neither was the case in this thread - > the guy claimed that he actually did something about the GIL, and > now we are all waiting for him to also tell us what it is that he > did. Ok, I did not remove the GIL, but I found a way to remove its notorious side effect on SMPs. So I am going to reveal it now. Forgive my strange sence of humor at 3 o'clock in the morning. First, think about this: (1) What is the GIL? (2) Where does it live? (3) How can it be manually released? The answer to this is: (1) A critical section (a lock/mutex object) (2) As a global object in Python25.dll on my computer (3) Using Python's C API or calling methods in a ctypes.CDLL object The Python C API has the ability to embed Python interpreters. You do this by importing Python25.dll into the process. ctypes has the ability to call functions in a DLL. So is it possible to embed Python in Python? And what would be consequence be? First, if I try to load a DLL more than once, Windows will detect this and just give me a handle to the currently imported library. So by importing Python25.dll with ctypes, I can just make my Python interpreter talk to itself through its own CAPI. Yes I can create sub interpreters using PyNew_Interpreter, but they all share the same GIL, so it's not useful here. So here is what I suddendly realized, and please don't laugh, its so simple its almost silly: I make some copies of Python25.dll, and call them Python25-1.dll, Python25-2.dll, Python25-3.dll, Python25-4.dll, etc. Then I load them all into my current Python process as ctypes.CDLL objects. Tada! I now have a pool of independent Python interpreters, not sharing the GIL, but still living in the same process. I can make the different interpreters talk to each other, including manipulating each other's GIL, using the using ctypes and Python's C API for embedding. So why does this circumvent the GIL? Because ctypes releases it before calling functions form a CDLL object. If I use my main interpreter to delegate a task to one of its embedded 'children', its GIL will be released while it is waiting for the answer. Associating each embedded interpreter with a threading.Thread is all that remains. The GIL is released while the thread operating the child interpreter is blocked. An there you have the answer. It's really very simple :-) Regards, Sturla Molden -- http://mail.python.org/mailman/listinfo/python-list
