>Number: 178301 >Category: ports >Synopsis: [patch] lang/python2[67]: SEM option is inconsistent >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-python >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Thu May 02 19:40:01 UTC 2013 >Closed-Date: >Last-Modified: >Originator: Jan Beich >Release: FreeBSD 10.0-CURRENT amd64 >Organization: >Environment: SEM unset (default, testing) sem.ko unloaded (default, testing)
http://svnweb.freebsd.org/changeset/base/201546 http://svnweb.freebsd.org/changeset/base/212852 >Description: python27 is always built with multiprocessing support on >= 9.0 or on 7.1..8.4 with sem.ko preloaded because autoconf finds working sem_open(). And python2[67] always USE_SEMAPHORES for locking which seems to work fine with libthr's sem_init(pshared=0). >How-To-Repeat: FreeBSD 9.0 or later: $ python3.3 -c 'import multiprocessing.synchronize' $ python2.7 -c 'import multiprocessing.synchronize' $ python2.6 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. FreeBSD 7.2..8.4: $ python3.3 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "/usr/local/lib/python3.3/multiprocessing/synchronize.py", line 27, in <module> from _multiprocessing import SemLock ImportError: cannot import name SemLock During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python3.3/multiprocessing/synchronize.py", line 32, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. $ python2.7 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line 59, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. $ python2.6 -c 'import multiprocessing.synchronize' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/lib/python2.6/multiprocessing/synchronize.py", line 33, in <module> " function, see issue 3770.") ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. >Fix: Just an incomplete attempt to make SEM more predictable: - remove checks for EOL releases - limit sem.ko warning and extra patches to 8.x users - warn 9.x users semaphores support cannot be disabled like in python3x - rely on autoconf to find sem_open et al. like netbsd or python3x - enable sem_open for python26 on >= 9.0 by default like python27 - don't use semaphores for locking if SEM is unset on 8.x --- python_sem.diff begins here --- Index: lang/python26/Makefile =================================================================== --- lang/python26/Makefile (revision 317131) +++ lang/python26/Makefile (working copy) @@ -57,16 +57,13 @@ OPTIONS_SINGLE= UCS OPTIONS_SINGLE_UCS= UCS2 UCS4 NLS_DESC= Enable Gettext support for the locale module +SEM_DESC= POSIX semaphores support (cannot be disabled on >= 9.0) .include <bsd.port.pre.mk> -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 SEM_MSG= "" .else -IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later -.endif # ${OSVERSION} >= 701106 -.else SEM_MSG= "@comment " .endif @@ -190,11 +187,10 @@ pre-patch: .endif post-patch: -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-setup.py + @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-Python_thread__pthread.h .endif -.endif .if defined(WITHOUT_NIS) ${REINPLACE_CMD} -e \ 's/disabled_module_list =[^]]*/&, "nis"/' \ Index: lang/python26/files/extra-patch-Python_thread__pthread.h =================================================================== --- lang/python26/files/extra-patch-Python_thread__pthread.h (revision 0) +++ lang/python26/files/extra-patch-Python_thread__pthread.h (working copy) @@ -0,0 +1,11 @@ +--- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 ++++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 +@@ -44,7 +44,7 @@ + #ifdef _POSIX_SEMAPHORES + /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so + we need to add 0 to make it work there as well. */ +-#if (_POSIX_SEMAPHORES+0) == -1 ++#if !defined(__FreeBSD__) && (_POSIX_SEMAPHORES+0) == -1 + #define HAVE_BROKEN_POSIX_SEMAPHORES + #else + #include <semaphore.h> Index: lang/python26/files/extra-patch-setup.py =================================================================== --- lang/python26/files/extra-patch-setup.py (revision 317131) +++ lang/python26/files/extra-patch-setup.py (working copy) @@ -1,26 +1,25 @@ --- setup.py.1 2009-03-12 04:07:36.000000000 +0000 +++ setup.py 2009-03-12 04:08:36.000000000 +0000 -@@ -1265,13 +1265,21 @@ +@@ -1385,20 +1385,12 @@ ) libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6'): +- elif platform in ('freebsd9', 'freebsd10'): +- macros = dict( # at least FreeBSD 9 +- HAVE_SEM_OPEN=1, +- HAVE_SEM_TIMEDWAIT=1, +- HAVE_FD_TRANSFER=1, +- ) +- libraries = [] +- + elif platform.startswith('freebsd'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) -- macros = dict( # FreeBSD -+ macros = dict( # FreeBSD 4-6 - HAVE_SEM_OPEN=0, - HAVE_SEM_TIMEDWAIT=0, - HAVE_FD_TRANSFER=1, -+ ) -+ libraries = [] -+ -+ elif platform in ('freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ macros = dict( # FreeBSD 7+ + macros = dict( # FreeBSD +- HAVE_SEM_OPEN=0, +- HAVE_SEM_TIMEDWAIT=0, + HAVE_SEM_OPEN=1, + HAVE_SEM_TIMEDWAIT=1, -+ HAVE_FD_TRANSFER=1, + HAVE_FD_TRANSFER=1, ) libraries = [] - Index: lang/python26/files/patch-Python_thread__pthread.h =================================================================== --- lang/python26/files/patch-Python_thread__pthread.h (revision 317131) +++ lang/python26/files/patch-Python_thread__pthread.h (working copy) @@ -1,33 +1,5 @@ --- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 +++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 -@@ -26,13 +26,18 @@ - #endif - #endif - -+#ifdef __FreeBSD__ -+#include <osreldate.h> -+#endif -+ - /* The POSIX spec says that implementations supporting the sem_* - family of functions must indicate this by defining - _POSIX_SEMAPHORES. */ - #ifdef _POSIX_SEMAPHORES - /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so - we need to add 0 to make it work there as well. */ --#if (_POSIX_SEMAPHORES+0) == -1 -+#if defined(__FreeBSD__) && __FreeBSD_version < 701104 && \ -+ (_POSIX_SEMAPHORES+0) == -1 - #define HAVE_BROKEN_POSIX_SEMAPHORES - #else - #include <semaphore.h> -@@ -44,7 +49,6 @@ - in default setting. So the process scope is preferred to get - enough number of threads to work. */ - #ifdef __FreeBSD__ --#include <osreldate.h> - #if __FreeBSD_version >= 500000 && __FreeBSD_version < 504101 - #undef PTHREAD_SYSTEM_SCHED_SUPPORTED - #endif @@ -149,6 +153,7 @@ { pthread_t th; Index: lang/python26/files/patch-setup.py =================================================================== --- lang/python26/files/patch-setup.py (revision 317131) +++ lang/python26/files/patch-setup.py (working copy) @@ -68,12 +68,20 @@ libraries = curses_libs) ) else: missing.append('_curses') -@@ -1381,7 +1385,7 @@ +@@ -1381,7 +1385,15 @@ ) libraries = [] - elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): ++ elif platform in ('freebsd9', 'freebsd10'): ++ macros = dict( # at least FreeBSD 9 ++ HAVE_SEM_OPEN=1, ++ HAVE_SEM_TIMEDWAIT=1, ++ HAVE_FD_TRANSFER=1, ++ ) ++ libraries = [] ++ ++ elif platform.startswith('freebsd'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) macros = dict( # FreeBSD Index: lang/python27/Makefile =================================================================== --- lang/python27/Makefile (revision 317131) +++ lang/python27/Makefile (working copy) @@ -57,16 +57,13 @@ OPTIONS_SINGLE= UCS OPTIONS_SINGLE_UCS= UCS2 UCS4 NLS_DESC= Enable Gettext support for the locale module +SEM_DESC= POSIX semaphores support (cannot be disabled on >= 9.0) .include <bsd.port.pre.mk> -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 SEM_MSG= "" .else -IGNORE= POSIX semaphore support only works in FreeBSD 7-STABLE and later -.endif # ${OSVERSION} >= 701106 -.else SEM_MSG= "@comment " .endif @@ -205,11 +202,10 @@ pre-patch: .endif post-patch: -.if ${PORT_OPTIONS:MSEM} -.if ${OSVERSION} >= 701106 +.if ${PORT_OPTIONS:MSEM} && ${OSVERSION} < 900007 @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-setup.py + @cd ${WRKSRC} && ${PATCH} ${PATCH_ARGS} < ${PATCHDIR}/extra-patch-Python_thread__pthread.h .endif -.endif .if defined(WITHOUT_NIS) ${REINPLACE_CMD} -e \ 's/disabled_module_list =[^]]*/&, "nis"/' \ Index: lang/python27/files/extra-patch-Python_thread__pthread.h =================================================================== --- lang/python27/files/extra-patch-Python_thread__pthread.h (revision 0) +++ lang/python27/files/extra-patch-Python_thread__pthread.h (working copy) @@ -0,0 +1,11 @@ +--- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 ++++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 +@@ -44,7 +44,7 @@ + #ifdef _POSIX_SEMAPHORES + /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so + we need to add 0 to make it work there as well. */ +-#if (_POSIX_SEMAPHORES+0) == -1 ++#if !defined(__FreeBSD__) && (_POSIX_SEMAPHORES+0) == -1 + #define HAVE_BROKEN_POSIX_SEMAPHORES + #else + #include <semaphore.h> Index: lang/python27/files/extra-patch-setup.py =================================================================== --- lang/python27/files/extra-patch-setup.py (revision 317131) +++ lang/python27/files/extra-patch-setup.py (working copy) @@ -1,23 +1,15 @@ --- setup.py.bak 2010-08-15 14:57:00.347134100 +0800 +++ setup.py 2010-08-15 15:00:06.019643300 +0800 -@@ -1402,10 +1402,22 @@ +@@ -1432,10 +1436,14 @@ class PyBuildExt(build_ext): macros = dict() libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6'): +- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): ++ elif platform in ('freebsd7', 'freebsd8'): # FreeBSD's P1003.1b semaphore support is very experimental # and has many known problems. (as of June 2008) - macros = dict() -+ macros = dict( # FreeBSD 4-6 -+ HAVE_SEM_OPEN=0, -+ HAVE_SEM_TIMEDWAIT=0, -+ HAVE_FD_TRANSFER=1, -+ ) -+ libraries = [] -+ -+ elif platform in ('freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): -+ macros = dict( # FreeBSD 7+ ++ macros = dict( # at least FreeBSD 7 + HAVE_SEM_OPEN=1, + HAVE_SEM_TIMEDWAIT=1, + HAVE_FD_TRANSFER=1, Index: lang/python27/files/patch-Python_thread__pthread.h =================================================================== --- lang/python27/files/patch-Python_thread__pthread.h (revision 317131) +++ lang/python27/files/patch-Python_thread__pthread.h (working copy) @@ -1,33 +1,5 @@ --- Python/thread_pthread.h.orig 2010-05-09 22:46:46.000000000 +0800 +++ Python/thread_pthread.h 2010-08-15 14:27:51.886823397 +0800 -@@ -26,13 +26,18 @@ - #endif - #endif - -+#ifdef __FreeBSD__ -+#include <osreldate.h> -+#endif -+ - /* The POSIX spec says that implementations supporting the sem_* - family of functions must indicate this by defining - _POSIX_SEMAPHORES. */ - #ifdef _POSIX_SEMAPHORES - /* On FreeBSD 4.x, _POSIX_SEMAPHORES is defined empty, so - we need to add 0 to make it work there as well. */ --#if (_POSIX_SEMAPHORES+0) == -1 -+#if defined(__FreeBSD__) && __FreeBSD_version < 701104 && \ -+ (_POSIX_SEMAPHORES+0) == -1 - #define HAVE_BROKEN_POSIX_SEMAPHORES - #else - #include <semaphore.h> -@@ -44,7 +49,6 @@ - in default setting. So the process scope is preferred to get - enough number of threads to work. */ - #ifdef __FreeBSD__ --#include <osreldate.h> - #if __FreeBSD_version >= 500000 && __FreeBSD_version < 504101 - #undef PTHREAD_SYSTEM_SCHED_SUPPORTED - #endif @@ -149,6 +153,7 @@ { pthread_t th; Index: lang/python27/files/patch-setup.py =================================================================== --- lang/python27/files/patch-setup.py (revision 317131) +++ lang/python27/files/patch-setup.py (working copy) @@ -68,15 +68,17 @@ libraries = curses_libs) ) else: missing.append('_curses') -@@ -1432,7 +1436,7 @@ +@@ -1438,6 +1442,10 @@ macros = dict() libraries = [] -- elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8'): -+ elif platform in ('freebsd4', 'freebsd5', 'freebsd6', 'freebsd7', 'freebsd8', 'freebsd9', 'freebsd10'): - # FreeBSD's P1003.1b semaphore support is very experimental - # and has many known problems. (as of June 2008) ++ elif platform.startswith('freebsd'): ++ macros = dict() ++ libraries = [] ++ + elif platform.startswith('openbsd'): macros = dict() + libraries = [] @@ -1484,7 +1488,7 @@ missing.append('linuxaudiodev') --- python_sem.diff ends here --- >Release-Note: >Audit-Trail: >Unformatted: _______________________________________________ freebsd-python@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-python To unsubscribe, send any mail to "freebsd-python-unsubscr...@freebsd.org"
