New submission from Paul Hobbs:
Using pid namespacing it is possible to have multiple processes with the same
pid. "semlock_new" creates a semaphore file with the template
"/dev/shm/mp{pid}-{counter}". This can conflict if the same semaphore file
already exists due to another Python process have the same pid.
This bug has been fixed in Python3: https://bugs.python.org/issue8713.
However, that patch is very large (40 files, ~4.4k changed lines) and only
incidentally fixes this bug while introducing a large backwards-incompatible
refactoring and feature addition.
The following small patch to just _multiprocessing/semaphore.c fixes the
problem by using the system clock and retrying to avoid conflicts:
--- a/Modules/_multiprocessing/semaphore.c
+++ b/Modules/_multiprocessing/semaphore.c
@@ -7,6 +7,7 @@
*/
#include "multiprocessing.h"
+#include <time.h>
enum { RECURSIVE_MUTEX, SEMAPHORE };
@@ -419,7 +420,7 @@ semlock_new(PyTypeObject *type, PyObject *args, PyObject
*kwds)
{
char buffer[256];
SEM_HANDLE handle = SEM_FAILED;
- int kind, maxvalue, value;
+ int kind, maxvalue, value, try;
PyObject *result;
static char *kwlist[] = {"kind", "value", "maxvalue", NULL};
static int counter = 0;
@@ -433,10 +434,24 @@ semlock_new(PyTypeObject *type, PyObject *args, PyObject
*kwds)
return NULL;
}
- PyOS_snprintf(buffer, sizeof(buffer), "/mp%ld-%d", (long)getpid(),
counter++);
+ /* With pid namespaces, we may have multiple processes with the same pid.
+ * Instead of relying on the pid to be unique, we use the microseconds time
+ * to attempt to a unique filename. */
+ for (try = 0; try < 100; ++try) {
+ struct timespec tv;
+ long arbitrary = clock_gettime(CLOCK_REALTIME, &tv) ? 0 : tv.tv_nsec;
+ PyOS_snprintf(buffer, sizeof(buffer), "/mp%ld-%d-%ld",
+ (long)getpid(),
+ counter++,
+ arbitrary);
+ SEM_CLEAR_ERROR();
+ handle = SEM_CREATE(buffer, value, maxvalue);
+ if (handle != SEM_FAILED)
+ break;
+ else if (errno != EEXIST)
+ goto failure;
+ }
- SEM_CLEAR_ERROR();
- handle = SEM_CREATE(buffer, value, maxvalue);
/* On Windows we should fail if GetLastError()==ERROR_ALREADY_EXISTS */
if (handle == SEM_FAILED || SEM_GET_LAST_ERROR() != 0)
goto failure;
----------
messages: 244211
nosy: Paul Hobbs
priority: normal
severity: normal
status: open
title: OSError 17 due to _multiprocessing/semaphore.c assuming a one-to-one Pid
-> process mapping.
type: enhancement
versions: Python 2.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue24303>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
