On 2023-09-26 14:20, Peter Ebden via Python-list wrote:
Hi all,
I've been working on embedding Python and have an interesting case around
locking with PyEval_RestoreThread which wasn't quite doing what I expect,
hoping someone can explain what I should expect here.
I have a little example (I'm running this in parallel from two different
threads; I have some more C code for that but I don't think it's super
interesting):
void run_python(PyThreadState* thread) {
LOG("Restoring thread %p...", thread);
PyEval_RestoreThread(thread);
LOG("Restored thread %p", thread);
PyRun_SimpleString("import time; print('sleeping'); time.sleep(3.0)");
LOG("Saving thread...");
PyThreadState* saved_thread = PyEval_SaveThread();
LOG("Saved thread %p", saved_thread);
}
This produces output like
11:46:48.110058893: Restoring thread 0xabc480...
11:46:48.110121656: Restored thread 0xabc480
11:46:48.110166060: Restoring thread 0xabc480...
sleeping
11:46:48.110464194: Restored thread 0xabc480
sleeping
11:46:51.111307541: Saving thread...
11:46:51.111361075: Saved thread 0xabc480
11:46:51.113116633: Saving thread...
11:46:51.113177605: Saved thread 0xabc480
The thing that surprises me is that both threads seem to be able to pass
PyEval_RestoreThread before either reaches the corresponding
PyEval_SaveThread call, which I wasn't expecting to happen; I assumed that
since RestoreThread acquires the GIL, that thread state would remain locked
until it's released.
I understand that the system occasionally switches threads, which I guess
might well happen with that time.sleep() call, but I wasn't expecting the
same thread to become usable somewhere else. Maybe I am just confusing
things by approaching the same Python thread from multiple OS threads
concurrently and should be managing my own locking around that?
Storing the result of PyEval_SaveThread in a local variable looks wrong
to me.
In the source for the regex module, I release the GIL with
PyEval_SaveThread and save its result. Then, when I want to claim the
GIL, I pass that saved value to PyEval_RestoreThread.
You seem to be releasing the GIL and discarding the result, so which
thread are you resuming when you call PyEval_RestoreThread?
It looks like you're resuming the same thread twice. As it's already
resumed the second time, no wonder it's not blocking!
--
https://mail.python.org/mailman/listinfo/python-list
