[ python-Bugs-1096310 ] sys.__stdout__ doco isn't discouraging enough
Bugs item #1096310, was opened at 2005-01-05 10:18 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1096310&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Just van Rossum (jvr) Assigned to: Nobody/Anonymous (nobody) Summary: sys.__stdout__ doco isn't discouraging enough Initial Comment: sys.__stdout__ is quite often abused (two independent sightings on c.l.py yesterday): people use it to restore stdout instead of saving the previous stdout. The real intended use for __stdout__ isn't all that clear (I keep wondering myself why it's there in the first place; its use case must be quite obscure), but more importantly I think the sys docs should contain a warning that __stdout__ is *not* suitable to restore sys.stdout after redirection. I'd produce a patch if I knew how to better describe what sys.__stdout__ was *for*. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1096310&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1089632 ] _DummyThread() objects not freed from threading._active map
Bugs item #1089632, was opened at 2004-12-22 15:37 Message generated for change (Comment added) made by saravanand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1089632&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 >Status: Open Resolution: Wont Fix Priority: 5 Submitted By: saravanand (saravanand) Assigned to: Nobody/Anonymous (nobody) Summary: _DummyThread() objects not freed from threading._active map Initial Comment: Problem Background: === I have Python Server module (long running) which accepts calls from several Python Clients over socket interface and forwards the call to a C++ component. This C++ component gives the reponses back to Python Server in a separate thread(created by C++ module) via callback. In the Python Callback implementation, the responses are sent to client in a synchronised manner using Python primitive threading.Semaphore. This Synchronisation is required as the C++ component can deliver parallel responses in different C++ threads. Here, the Python Server creates the semaphore object per client when the client request arrives (in Python thread). This same object is acquired & released in the C++ callback thread(s). Here we observed that Windows Events are getting created whenever the acquire method is executed in the Python Callback implementation in the context of C++ thread. But the same event is not freed by the Python Interpreter even after the termination of the C++ thread. Because of this, a Windows Event handles are getting leaked in the Python Server. Problem Description: == When we checked the Python module threading.py, we found that, every time a non-python thread (in our case C++ created thread), enters python and accessesn a primitive in threading module (eg: Semaphore, RLock), python looks for an entry for this thread in the _active map using thread ID as the Key. Since no entry exists for such C++ created threads, a _DummyThread object is created and added to the _active map for this C++ thread. For every _DummyThread object that is created, there is a corresponding Windows Event also getting created. Since this entry is never removed from the _active map even after the termination of the C++ thread ( as we could make out from the code in threading.py),for every "unique" C++ thread that enters python, a Windows Event is allocated and this manifests as continuous increase in the Handle count in my Python server ( as seen in Windows PerfMon/Task Manager). Is there a way to avoid this caching in Python Interpreter? Why cant Python remove this entry from the map when the C++ thread terminates. Or if Python can't get to know about the thread termination, should it not implement some kind of Garbage collection for the entries in this Map (especially entries for the _DummyThread objects). Does this require a correction in Python modulethreading.py? or is this caching behaviour by design? -- >Comment By: saravanand (saravanand) Date: 2005-01-05 15:59 Message: Logged In: YES user_id=1181691 asdfas -- Comment By: Brett Cannon (bcannon) Date: 2004-12-24 08:05 Message: Logged In: YES user_id=357491 Yes, it is by design. If you read the source you will notice that the comment mentions that the _DummyThread object is flagged as a daemon thread and thus should not be expected to be killed. The comment also mentions how they are not garbage collected. As stated in the docs, dummy threads are of limited functionality. You could cheat and remove the entries yourself from threading._active, but that might not be future-safe. I would just make sure that all threads are created through the threading or thread module, even if it means creating a minimal wrapper in Python for your C++ code to call through that to execute your C++ threads. If you want the docs to be more specific please feel free to submit a patch for the docs. Or if you can come up with a good way for the dummy threads to clean up after themselves then you can also submit that. But since the source code specifies that this expected and the docs say that dummy threads are of limited functionality I am closing as "won't fix". -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1089632&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1089632 ] _DummyThread() objects not freed from threading._active map
Bugs item #1089632, was opened at 2004-12-22 15:37 Message generated for change (Comment added) made by saravanand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1089632&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: Wont Fix Priority: 5 Submitted By: saravanand (saravanand) Assigned to: Nobody/Anonymous (nobody) Summary: _DummyThread() objects not freed from threading._active map Initial Comment: Problem Background: === I have Python Server module (long running) which accepts calls from several Python Clients over socket interface and forwards the call to a C++ component. This C++ component gives the reponses back to Python Server in a separate thread(created by C++ module) via callback. In the Python Callback implementation, the responses are sent to client in a synchronised manner using Python primitive threading.Semaphore. This Synchronisation is required as the C++ component can deliver parallel responses in different C++ threads. Here, the Python Server creates the semaphore object per client when the client request arrives (in Python thread). This same object is acquired & released in the C++ callback thread(s). Here we observed that Windows Events are getting created whenever the acquire method is executed in the Python Callback implementation in the context of C++ thread. But the same event is not freed by the Python Interpreter even after the termination of the C++ thread. Because of this, a Windows Event handles are getting leaked in the Python Server. Problem Description: == When we checked the Python module threading.py, we found that, every time a non-python thread (in our case C++ created thread), enters python and accessesn a primitive in threading module (eg: Semaphore, RLock), python looks for an entry for this thread in the _active map using thread ID as the Key. Since no entry exists for such C++ created threads, a _DummyThread object is created and added to the _active map for this C++ thread. For every _DummyThread object that is created, there is a corresponding Windows Event also getting created. Since this entry is never removed from the _active map even after the termination of the C++ thread ( as we could make out from the code in threading.py),for every "unique" C++ thread that enters python, a Windows Event is allocated and this manifests as continuous increase in the Handle count in my Python server ( as seen in Windows PerfMon/Task Manager). Is there a way to avoid this caching in Python Interpreter? Why cant Python remove this entry from the map when the C++ thread terminates. Or if Python can't get to know about the thread termination, should it not implement some kind of Garbage collection for the entries in this Map (especially entries for the _DummyThread objects). Does this require a correction in Python modulethreading.py? or is this caching behaviour by design? -- >Comment By: saravanand (saravanand) Date: 2005-01-05 16:24 Message: Logged In: YES user_id=1181691 I tried the following workaround which is working (causes no handle leaks) Workaround is to change threading.semaphore to Windows Extension module APIs win32event.CreateMutex(), win32event.WaitForSingleObject and win32event.ReleaseMutex () After this change, there are no handle leaks. So, my questions are: 1) Is this workaround OK or are there any other issues regarding the win32api usage ? 2) you suggested to create minimal python wrappers for C++ code and call C++ from python (instead of C++ thread callbacks). So I would like to know, in general, whether it is a bad idea for c++ threads to callback into Python. If yes, what are the issues (apart from the handle leak mentioned before). If no, I would like to live with the above workaround. Thanks in advance -- Comment By: saravanand (saravanand) Date: 2005-01-05 15:59 Message: Logged In: YES user_id=1181691 asdfas -- Comment By: Brett Cannon (bcannon) Date: 2004-12-24 08:05 Message: Logged In: YES user_id=357491 Yes, it is by design. If you read the source you will notice that the comment mentions that the _DummyThread object is flagged as a daemon thread and thus should not be expected to be killed. The comment also mentions how they are not garbage collected. As stated in the docs, dummy threads are of limited functionality. You could cheat and remove the entries yourself from threading._active, but that might not be future-safe. I would just make sure that all threads are created through the threading or thread module, even if it means creating a minimal wrapper in Python for your C++ code to call through that to execute your C++ threads. If y
[ python-Feature Requests-1073198 ] Extension to optparse: options with facultative value
Feature Requests item #1073198, was opened at 2004-11-25 09:18 Message generated for change (Comment added) made by gward You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1073198&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: pollastri (pollastri) Assigned to: Greg Ward (gward) Summary: Extension to optparse: options with facultative value Initial Comment: When parsing command line options, I found very useful to have an option with a facultative value, able to do the following: 1-tell to me if the option was or was not seen on the command line, return the value None if the option was not seen; 2-if the option only was specified, return a default value. 3-if the option with a value was specified on the command line, return the specified value; A way to reach this goal can be the addition of a new value for the options actions in module optparse, it may be something like "store_with_default". -- >Comment By: Greg Ward (gward) Date: 2005-01-05 08:19 Message: Logged In: YES user_id=14422 I don't know what "facultative" means, but what you want is optional option arguments. Don't worry, I want that feature too, and have even started implementing it on a branch of the Optik source repository. See http://sourceforge.net/tracker/?func=detail&aid=1050431&group_id=38019&atid=421100 for a similar feature request. I'm leaving this open, since I haven't finished anything or merged it into the Python CVS. It could be a few months before there's any action here, and it certainly won't happen before Optik 1.6 / Python 2.5. Might require Optik 2.0, in which case I'm not certain when it'll get into Python. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1073198&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1089955 ] optparse .error() should print options list
Feature Requests item #1089955, was opened at 2004-12-22 13:53 Message generated for change (Comment added) made by gward You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1089955&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Mike Orr (hierro) Assigned to: Greg Ward (gward) Summary: optparse .error() should print options list Initial Comment: Why doesn't optparse.OptionParser.error() print the list of correct options along with the error message and usage? This is what the user needs to know whenever there's an error. One can override this in a subclass, but it's frustrating to have to do it every time. To do this, change the first line in OptionParser.error() from: self.print_usage(sys.stderr) to: self.print_help(sys.stderr) print >>sys.stderr -- >Comment By: Greg Ward (gward) Date: 2005-01-05 08:20 Message: Logged In: YES user_id=14422 Because very few conventional Unix programs act this way: $ -z ls: invalid option -- z Try `ls --help' for more information. $ tar -a tar: invalid option -- a Try `tar --help' for more information. $ cp -z cp: invalid option -- z Try `cp --help' for more information. Note the pattern. (Hmmm: perhaps Optik should throw in a "try --help" if there's a help option in the current parser.) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1089955&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1071597 ] configure problem on HP-UX 11.11
Bugs item #1071597, was opened at 2004-11-23 10:30 Message generated for change (Comment added) made by harripasanen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1071597&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Harri Pasanen (harripasanen) Assigned to: Nobody/Anonymous (nobody) Summary: configure problem on HP-UX 11.11 Initial Comment: Python 2.4c1 has this problem (but if I recall, so did 2.3.3)_ Using gcc 3.3.3 to build on HP-UX 11.11, the configure out of the box is a bit off, resulting in a failed build, due to missing thread symbols: /usr/ccs/bin/ld: Unsatisfied symbols: PyThread_acquire_lock (first referenced in libpython2.4.a(import.o)) (code) PyThread_exit_thread (first referenced in libpython2.4.a(threadmodule.o)) (code) PyThread_allocate_lock (first referenced in libpython2.4.a(import.o)) (code) PyThread_free_lock (first referenced in libpython2.4.a(threadmodule.o)) (code) PyThread_start_new_thread (first referenced in libpython2.4.a(threadmodule.o)) (code) PyThread_release_lock (first referenced in libpython2.4.a(import.o)) (code) PyThread_get_thread_ident (first referenced in libpython2.4.a(import.o)) (code) PyThread__init_thread (first referenced in libpython2.4.a(thread.o)) (code) collect2: ld returned 1 exit status A workaround is to manually edit pyconfig.h, adding #define _POSIX_THREADS (The reason it is not picked up is that unistd.h on HP-UX has this comment: / * The following defines are specified in the standard, but are not yet * implemented: * *_POSIX_THREADS can't be defined until all * features are implemented ) The implementation seems however to be sufficiently complete to permit compiling and running Python with _POSIX_THREADS. While I'm editing pyconfig.h, I also comment out _POSIX_C_SOURCE definition, as it will result in lots of compilation warnings, of the style: gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I../Python-2.4c1/Include -DPy_BUILD_CORE -o Objects/frameobject.o ../Python-2.4c1/Objects/frameobject.c In file included from ../Python-2.4c1/Include/Python.h:8, from ../Python-2.4c1/Objects/frameobject.c:4: pyconfig.h:835:1: warning: "_POSIX_C_SOURCE" redefined :6:1: warning: this is the location of the previous definition So, to recapitulate: After configure, add #define _POSIX_THREADS and comment out #define _POSIX_C_SOURCE 200112L That will give you a Python working rather well, with "make test" producing: 251 tests OK. 1 test failed: test_pty 38 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_gdbm test_gl test_imgfile test_largefile test_linuxaudiodev test_locale test_macfs test_macostools test_nis test_normalization test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_sunaudiodev test_tcl test_timeout test_urllib2net test_urllibnet test_winreg test_winsound 1 skip unexpected on hp-ux11: test_tcl -- >Comment By: Harri Pasanen (harripasanen) Date: 2005-01-05 14:46 Message: Logged In: YES user_id=77088 _POSIX_C_SOURCE appears to come from the gcc driver, along with a few other macros. If I add "-v" to compilation, the output expands to: Reading specs from /usr/local/lib/gcc-lib/hppa2.0w-hp-hpux11.11/3.3.3/specs Configured with: /scratch/zack/pkgbuild/3.3.1/hpux-11/gcc-3.3.3/configure --enable-languages=c,c++ --enable-threads=posix --disable-nls --with-gnu-as --without-gnu-ld --with-as=/usr/local/bin/as --prefix=/usr/local Thread model: posix gcc version 3.3.3 /usr/local/lib/gcc-lib/hppa2.0w-hp-hpux11.11/3.3.3/cc1 -quiet -v -I. -I./Include -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_REENTRANT -D_THREAD_SAFE -D_POSIX_C_SOURCE=199506L -DNDEBUG -DPy_BUILD_CORE Modules/python.c -quiet -dumpbase python.c -auxbase-strip Modules/python.o -g -O3 -Wall -Wstrict-prototypes -version -fno-strict-aliasing -o /var/tmp//ccLnAhZ0.s GNU C version 3.3.3 (hppa2.0w-hp-hpux11.11) compiled by GNU C version 3.3.3. GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072 ignoring nonexistent directory "/usr/local/hppa2.0w-hp-hpux11.11/include" #include "..." search starts here: #include <...> search starts here: . Include /usr/local/include /usr/local/lib/gcc-lib/hppa2.0w-hp-hpux11.11/3.3.3/include /usr/include End of sea
[ python-Bugs-1095802 ] "Macintosh" references in the docs need to be checked.
Bugs item #1095802, was opened at 2005-01-04 07:14 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1095802&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jack Jansen (jackjansen) Assigned to: Nobody/Anonymous (nobody) Summary: "Macintosh" references in the docs need to be checked. Initial Comment: As of 2.4 Python no longer runs on MacOS9 or earlier. We need to go through all the documentation loooking for references to "macintosh", "mac", "apple", "macos" and such to check whether these references are still valid. -- >Comment By: Brett Cannon (bcannon) Date: 2005-01-05 23:21 Message: Logged In: YES user_id=357491 Should we change the mentions to "OS X" or "darwin"? It would help remove any possible misunderstanding of OS 9 support and make sure we caught all doc mentions. Could also help specify and difference between Darwin and OS X explicitly. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1095802&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
