[ python-Bugs-1603424 ] subprocess.py (py2.5) wrongly claims py2.2 compatibility
Bugs item #1603424, was opened at 2006-11-27 02:07 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1603424&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Tim Wegener (twegener) Assigned to: Peter Åstrand (astrand) Summary: subprocess.py (py2.5) wrongly claims py2.2 compatibility Initial Comment: From the comments in subprocess.py (py2.5): # This module should remain compatible with Python 2.2, see PEP 291. However, using it from Python 2.2 gives: NameError: global name 'set' is not defined (set built-in used on line 1005) The subprocess.py in py2.4 was 2.2 compatible. Either the compatibility comment should be removed/amended or compatibility fixed. -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-07 10:01 Message: Logged In: YES user_id=344921 Originator: NO Fixed in revision 53293 (trunk) and 53294 (2.5). -- Comment By: Robert Carr (racarr) Date: 2006-12-05 16:10 Message: Logged In: YES user_id=1649655 Originator: NO Index: subprocess.py === --- subprocess.py (revision 52918) +++ subprocess.py (working copy) @@ -1004,8 +1004,8 @@ # Close pipe fds. Make sure we don't close the same # fd more than once, or standard fds. -for fd in set((p2cread, c2pwrite, errwrite))-set((0,1,2)): -if fd: os.close(fd) +for fd in (p2cread,c2pwrite,errwrite): +if fd not in (0,1,2): os.close(fd) # Close all other fds, if asked for if close_fds: Fixed? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1603424&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-539444 ] asyncore file wrapper & os.error
Bugs item #539444, was opened at 2002-04-04 22:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Hylton (jhylton) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore file wrapper & os.error Initial Comment: The file wrapper makes a file descriptor look like an asycnore socket. When its recv() method is invoked, it calls os.read(). I use this in an application where os.read() occasionally raises os.error (11, 'Resource temporarily unavailable'). I think that asyncore should catch this error and treat it just like EWOULDBLOCK. But I'd like a second opinion. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 11:45 Message: Logged In: YES user_id=21627 Originator: NO Notice that the ZODB issue is marked as fixed. I would like to know how that was fixed, and I still would like to know what operating system this problem occurred on. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 07:00 Message: Logged In: YES user_id=341410 Originator: NO I don't see an issue with treating EAGAIN as EWOULDBLOCK. In the cases where EAGAIN != EWOULDBLOCK (in terms of constant value), treating them the same would be the right thing. In the case where the values were the same, nothing would change. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-07 11:03 Message: Logged In: YES user_id=21627 I'm still uncertain what precisely was happening here. What system was this on? On many systems, EAGAIN is EWOULDBLOCK; if that is the case, adding EAGAIN to the places that currently handle EWOULDBLOCK won't change anything. -- Comment By: Jeremy Hylton (jhylton) Date: 2002-04-05 18:44 Message: Logged In: YES user_id=31392 It happens when the file is a pipe. For details, see the ZEO bug report at https://sourceforge.net/tracker/index.php? func=detail&aid=536416&group_id=15628&atid=115628 I've included the traceback from that bug report, too. error: uncaptured python exception, closing channel(exceptions.OSError:[Errno 11] Resource temporarily unavailable [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|poll|92] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|handle_read_event|386] [/home/zope/opt/Python-2.1.2/lib/python2.1/site- packages/ZEO/trigger.py|handle_read|95] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|338] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|520]) Exception exceptions.OSError: (9, 'Bad file descriptor') in ignored -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 11:00 Message: Logged In: YES user_id=21627 Can you report details of the file that returns EWOULDBLOCK? This is not supposed to happen in applications of the file_wrapper. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1615376 ] subprocess doesn\'t handle SIGPIPE
Feature Requests item #1615376, was opened at 2006-12-14 01:21 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1615376&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Python Library >Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mark Diekhans (diekhans) Assigned to: Peter Åstrand (astrand) Summary: subprocess doesn\'t handle SIGPIPE Initial Comment: subprocess keeps other side of child pipe open, making use of SIGPIPE to terminate writers in a pipeline not possible. This is probably a matter of documentation or providing a method to link up processes, as the parent end of the pipe must remain open until it is connected to the next process in the pipeline. An option to enable sigpipe in child would be nice. Simple example attached. -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:01 Message: Logged In: YES user_id=344921 Originator: NO One easy solution is to simply close the pipe in the parent after starting both processes, before calling p1.wait(): p1.stdout.close() It's not "perfect", though, p1 will execute a while before recieving SIGPIPE. For a perfect solution, it would be necessary to close the pipe end in the parent after the fork but before the exec in the child. This would require some kind of synchronization. Moving to feature request. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1615376&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1604851 ] subprocess.Popen closes fds for sys.stdout or sys.stderr
Bugs item #1604851, was opened at 2006-11-28 23:17 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1604851&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Duplicate Priority: 5 Private: No Submitted By: Nishkar Grover (ngrover) Assigned to: Peter Åstrand (astrand) Summary: subprocess.Popen closes fds for sys.stdout or sys.stderr Initial Comment: I found a problem in subprocess.Popen's _execute_child() method for POSIX, where the child process will close the fds for sys.stdout and/or sys.stderr if I use those as stdout and/or stderr when creating a subprocess.Popen object. Here's what I saw by default when using the 2.4.4 version of Python... % ./python Python 2.4.4 (#1, Nov 28 2006, 14:08:29) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import sys, subprocess >>> uname = subprocess.Popen('uname -a', shell=True, stdout=sys.stdout) >>> uname: write error: Bad file descriptor >>> Then, I updated subprocess.py and made the following changes... % diff subprocess.py subprocess.py.orig 924c924 < # fd more than once and don't close sys.stdout or sys.stderr. --- > # fd more than once. 927c927 < if c2pwrite and c2pwrite not in (p2cread, sys.stdout.fileno(), sys.stderr.fileno()): --- > if c2pwrite and c2pwrite not in (p2cread,): 929c929 < if errwrite and errwrite not in (p2cread, c2pwrite, sys.stdout.fileno(), sys.stderr.fileno()): --- > if errwrite and errwrite not in (p2cread, c2pwrite): After that, I saw the following... % ./python Python 2.4.4 (#1, Nov 28 2006, 14:08:29) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import sys, subprocess >>> uname = subprocess.Popen('uname -a', shell=True, stdout=sys.stdout) >>> Linux schnauzer 2.6.9-42.0.2.ELsmp #1 SMP Thu Aug 17 18:00:32 EDT 2006 i686 >>> i686 i386 GNU/Linux >>> I'm attaching the modified version of subprocess.py. Please consider adding this fix to future versions of Python. Thanks! -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:05 Message: Logged In: YES user_id=344921 Originator: NO Duplicate of 1531862. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1604851&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1590864 ] subprocess deadlock
Bugs item #1590864, was opened at 2006-11-05 17:06 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Tsai (michaeltsai) Assigned to: Peter Åstrand (astrand) Summary: subprocess deadlock Initial Comment: When I use subprocess.py from a child thread, sometimes it deadlocks. I determined that the new process is blocked during an import: #0 0x90024427 in semaphore_wait_signal_trap () #1 0x90028414 in pthread_cond_wait () #2 0x004c77bf in PyThread_acquire_lock (lock=0x3189a0, waitflag=1) at Python/thread_pthread.h:452 #3 0x004ae2a6 in lock_import () at Python/import.c:266 #4 0x004b24be in PyImport_ImportModuleLevel (name=0xaad74 "errno", globals=0xbaed0, locals=0x502aa0, fromlist=0xc1378, level=-1) at Python/import.c:2054 #5 0x0048d2e2 in builtin___import__ (self=0x0, args=0x53724c90, kwds=0x0) at Python/bltinmodule.c:47 #6 0x0040decb in PyObject_Call (func=0xa94b8, arg=0x53724c90, kw=0x0) at Objects/abstract.c:1860 and that the code in question is in os.py: def _execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR I think the problem is that since exec (the C function) hasn't yet been called in the new process, it's inherited from the fork a lock that's already held. The main process will eventually release its copy of the lock, but this will not unlock it in the new process, so it deadlocks. If I change os.py so that it imports the constants outside of _execvpe, the new process no longer blocks in this way. This is on Mac OS X 10.4.8. -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:10 Message: Logged In: YES user_id=344921 Originator: NO Can you provide a test case or sample code that demonstrates this problem? I'm a bit unsure of if this really is a subprocess bug or a more general Python bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1598181 ] subprocess.py: O(N**2) bottleneck
Bugs item #1598181, was opened at 2006-11-17 07:40 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1598181&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open >Resolution: Fixed Priority: 5 Private: No Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Peter Åstrand (astrand) Summary: subprocess.py: O(N**2) bottleneck Initial Comment: subprocess.py (Python 2.5, current SVN, probably all versions) contains this O(N**2) code: bytes_written = os.write(self.stdin.fileno(), input[:512]) input = input[bytes_written:] For large but reasonable "input" the second line is rate limiting. Luckily, it is very easy to remove this bottleneck. I'll upload a simple patch. Below is a small script that demonstrates the huge speed difference. The output on my machine is: creating input 0.888417959213 slow slicing input 61.1553330421 creating input 0.863168954849 fast slicing input 0.0163860321045 done The numbers are times in seconds. This is the source: import time import sys size = 100 t0 = time.time() print "creating input" input = "\n".join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print "slow slicing input" n_out_slow = 0 while True: out = input[:512] n_out_slow += 1 input = input[512:] if not input: break print time.time()-t0 t0 = time.time() print "creating input" input = "\n".join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print "fast slicing input" n_out_fast = 0 input_done = 0 while True: out = input[input_done:input_done+512] n_out_fast += 1 input_done += 512 if input_done >= len(input): break print time.time()-t0 assert n_out_fast == n_out_slow print "done" -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:36 Message: Logged In: YES user_id=344921 Originator: NO Fixed in trunk revision 53295. Is this a good candidate for backporting to 25-maint? -- Comment By: Mike Klaas (mklaas) Date: 2007-01-04 19:20 Message: Logged In: YES user_id=1611720 Originator: NO I reviewed the patch--the proposed fix looks good. Minor comments: - "input_done" sounds like a flag, not a count of written bytes - buffer() could be used to avoid the 512-byte copy created by the slice -- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2006-11-17 07:43 Message: Logged In: YES user_id=71407 Originator: YES Sorry, I didn't know the tracker would destroy the indentation. I'm uploading the demo source as a separate file. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1598181&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1598181 ] subprocess.py: O(N**2) bottleneck
Bugs item #1598181, was opened at 2006-11-16 22:40 Message generated for change (Comment added) made by rwgk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1598181&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: Fixed Priority: 5 Private: No Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Peter Åstrand (astrand) Summary: subprocess.py: O(N**2) bottleneck Initial Comment: subprocess.py (Python 2.5, current SVN, probably all versions) contains this O(N**2) code: bytes_written = os.write(self.stdin.fileno(), input[:512]) input = input[bytes_written:] For large but reasonable "input" the second line is rate limiting. Luckily, it is very easy to remove this bottleneck. I'll upload a simple patch. Below is a small script that demonstrates the huge speed difference. The output on my machine is: creating input 0.888417959213 slow slicing input 61.1553330421 creating input 0.863168954849 fast slicing input 0.0163860321045 done The numbers are times in seconds. This is the source: import time import sys size = 100 t0 = time.time() print "creating input" input = "\n".join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print "slow slicing input" n_out_slow = 0 while True: out = input[:512] n_out_slow += 1 input = input[512:] if not input: break print time.time()-t0 t0 = time.time() print "creating input" input = "\n".join([str(i) for i in xrange(size)]) print time.time()-t0 t0 = time.time() print "fast slicing input" n_out_fast = 0 input_done = 0 while True: out = input[input_done:input_done+512] n_out_fast += 1 input_done += 512 if input_done >= len(input): break print time.time()-t0 assert n_out_fast == n_out_slow print "done" -- >Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2007-01-07 07:15 Message: Logged In: YES user_id=71407 Originator: YES Thanks for the fixes! -- Comment By: Peter Åstrand (astrand) Date: 2007-01-07 06:36 Message: Logged In: YES user_id=344921 Originator: NO Fixed in trunk revision 53295. Is this a good candidate for backporting to 25-maint? -- Comment By: Mike Klaas (mklaas) Date: 2007-01-04 10:20 Message: Logged In: YES user_id=1611720 Originator: NO I reviewed the patch--the proposed fix looks good. Minor comments: - "input_done" sounds like a flag, not a count of written bytes - buffer() could be used to avoid the 512-byte copy created by the slice -- Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2006-11-16 22:43 Message: Logged In: YES user_id=71407 Originator: YES Sorry, I didn't know the tracker would destroy the indentation. I'm uploading the demo source as a separate file. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1598181&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1590864 ] subprocess deadlock
Bugs item #1590864, was opened at 2006-11-05 11:06 Message generated for change (Comment added) made by michaeltsai You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Tsai (michaeltsai) Assigned to: Peter Åstrand (astrand) Summary: subprocess deadlock Initial Comment: When I use subprocess.py from a child thread, sometimes it deadlocks. I determined that the new process is blocked during an import: #0 0x90024427 in semaphore_wait_signal_trap () #1 0x90028414 in pthread_cond_wait () #2 0x004c77bf in PyThread_acquire_lock (lock=0x3189a0, waitflag=1) at Python/thread_pthread.h:452 #3 0x004ae2a6 in lock_import () at Python/import.c:266 #4 0x004b24be in PyImport_ImportModuleLevel (name=0xaad74 "errno", globals=0xbaed0, locals=0x502aa0, fromlist=0xc1378, level=-1) at Python/import.c:2054 #5 0x0048d2e2 in builtin___import__ (self=0x0, args=0x53724c90, kwds=0x0) at Python/bltinmodule.c:47 #6 0x0040decb in PyObject_Call (func=0xa94b8, arg=0x53724c90, kw=0x0) at Objects/abstract.c:1860 and that the code in question is in os.py: def _execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR I think the problem is that since exec (the C function) hasn't yet been called in the new process, it's inherited from the fork a lock that's already held. The main process will eventually release its copy of the lock, but this will not unlock it in the new process, so it deadlocks. If I change os.py so that it imports the constants outside of _execvpe, the new process no longer blocks in this way. This is on Mac OS X 10.4.8. -- >Comment By: Michael Tsai (michaeltsai) Date: 2007-01-07 12:09 Message: Logged In: YES user_id=817528 Originator: YES I don't have time at the moment to write sample code that reproduces this. But, FYI, I was using PyObjC to create the threads. It might not happen with "threading" threads. And second, I think it's a bug in os.py, not in subprocess.py. Sorry for the confusion. -- Comment By: Peter Åstrand (astrand) Date: 2007-01-07 09:10 Message: Logged In: YES user_id=344921 Originator: NO Can you provide a test case or sample code that demonstrates this problem? I'm a bit unsure of if this really is a subprocess bug or a more general Python bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-539444 ] asyncore file wrapper & os.error
Bugs item #539444, was opened at 2002-04-04 12:57 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Hylton (jhylton) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore file wrapper & os.error Initial Comment: The file wrapper makes a file descriptor look like an asycnore socket. When its recv() method is invoked, it calls os.read(). I use this in an application where os.read() occasionally raises os.error (11, 'Resource temporarily unavailable'). I think that asyncore should catch this error and treat it just like EWOULDBLOCK. But I'd like a second opinion. -- >Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 10:03 Message: Logged In: YES user_id=341410 Originator: NO Jeremy Hylton states what he did to fix it in ZEO. In terms of platform, I would guess that this is likely linux, as multiple people seem to be able to reproduce the error, and you can't reliably use signals in Windows without killing the process. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 02:45 Message: Logged In: YES user_id=21627 Originator: NO Notice that the ZODB issue is marked as fixed. I would like to know how that was fixed, and I still would like to know what operating system this problem occurred on. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-06 22:00 Message: Logged In: YES user_id=341410 Originator: NO I don't see an issue with treating EAGAIN as EWOULDBLOCK. In the cases where EAGAIN != EWOULDBLOCK (in terms of constant value), treating them the same would be the right thing. In the case where the values were the same, nothing would change. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-07 01:03 Message: Logged In: YES user_id=21627 I'm still uncertain what precisely was happening here. What system was this on? On many systems, EAGAIN is EWOULDBLOCK; if that is the case, adding EAGAIN to the places that currently handle EWOULDBLOCK won't change anything. -- Comment By: Jeremy Hylton (jhylton) Date: 2002-04-05 08:44 Message: Logged In: YES user_id=31392 It happens when the file is a pipe. For details, see the ZEO bug report at https://sourceforge.net/tracker/index.php? func=detail&aid=536416&group_id=15628&atid=115628 I've included the traceback from that bug report, too. error: uncaptured python exception, closing channel(exceptions.OSError:[Errno 11] Resource temporarily unavailable [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|poll|92] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|handle_read_event|386] [/home/zope/opt/Python-2.1.2/lib/python2.1/site- packages/ZEO/trigger.py|handle_read|95] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|338] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|520]) Exception exceptions.OSError: (9, 'Bad file descriptor') in ignored -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 01:00 Message: Logged In: YES user_id=21627 Can you report details of the file that returns EWOULDBLOCK? This is not supposed to happen in applications of the file_wrapper. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-539444 ] asyncore file wrapper & os.error
Bugs item #539444, was opened at 2002-04-04 22:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Hylton (jhylton) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore file wrapper & os.error Initial Comment: The file wrapper makes a file descriptor look like an asycnore socket. When its recv() method is invoked, it calls os.read(). I use this in an application where os.read() occasionally raises os.error (11, 'Resource temporarily unavailable'). I think that asyncore should catch this error and treat it just like EWOULDBLOCK. But I'd like a second opinion. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 20:18 Message: Logged In: YES user_id=21627 Originator: NO Ok; still I wonder what the problem is. In the original report, Jeremy said "should catch this error and treat it just like EWOULDBLOCK". Now, EWOULDBLOCK is handled in dispatcher.connect, dispatcher.accept, and dispatcher.send - not in dispatcher.recv. So what would it help to treat EAGAIN the same way? -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 19:03 Message: Logged In: YES user_id=341410 Originator: NO Jeremy Hylton states what he did to fix it in ZEO. In terms of platform, I would guess that this is likely linux, as multiple people seem to be able to reproduce the error, and you can't reliably use signals in Windows without killing the process. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 11:45 Message: Logged In: YES user_id=21627 Originator: NO Notice that the ZODB issue is marked as fixed. I would like to know how that was fixed, and I still would like to know what operating system this problem occurred on. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 07:00 Message: Logged In: YES user_id=341410 Originator: NO I don't see an issue with treating EAGAIN as EWOULDBLOCK. In the cases where EAGAIN != EWOULDBLOCK (in terms of constant value), treating them the same would be the right thing. In the case where the values were the same, nothing would change. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-07 11:03 Message: Logged In: YES user_id=21627 I'm still uncertain what precisely was happening here. What system was this on? On many systems, EAGAIN is EWOULDBLOCK; if that is the case, adding EAGAIN to the places that currently handle EWOULDBLOCK won't change anything. -- Comment By: Jeremy Hylton (jhylton) Date: 2002-04-05 18:44 Message: Logged In: YES user_id=31392 It happens when the file is a pipe. For details, see the ZEO bug report at https://sourceforge.net/tracker/index.php? func=detail&aid=536416&group_id=15628&atid=115628 I've included the traceback from that bug report, too. error: uncaptured python exception, closing channel(exceptions.OSError:[Errno 11] Resource temporarily unavailable [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|poll|92] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|handle_read_event|386] [/home/zope/opt/Python-2.1.2/lib/python2.1/site- packages/ZEO/trigger.py|handle_read|95] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|338] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|520]) Exception exceptions.OSError: (9, 'Bad file descriptor') in ignored -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 11:00 Message: Logged In: YES user_id=21627 Can you report details of the file that returns EWOULDBLOCK? This is not supposed to happen in applications of the file_wrapper. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-415692 ] smarter temporary file object
Feature Requests item #415692, was opened at 2001-04-12 10:37 Message generated for change (Comment added) made by djmitche You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=415692&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: smarter temporary file object Initial Comment: Jim Fulton suggested the following: I wonder if it would be a good idea to have a new kind of temporary file that stored data in memory unless: - The data exceeds some size, or - Somebody asks for a fileno. Then the cgi module (and other apps) could use this thing in a uniform way. -- Comment By: Dustin J. Mitchell (djmitche) Date: 2007-01-07 14:36 Message: Logged In: YES user_id=7446 Originator: NO Patch is at http://python.org/sf/1630118 -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-01-02 22:52 Message: Logged In: YES user_id=6380 Originator: YES I've reopened the issue for you. Do try to interest some other core developer in reviewing your code, or it will take a long time... Thanks for remembering! -- Comment By: Dustin J. Mitchell (djmitche) Date: 2007-01-02 22:30 Message: Logged In: YES user_id=7446 Originator: NO I have a potential implementation for this, intended to be included in Lib/tempfile.py. Because the issue is closed, I can't attach it. Let's see if posting to the issue will open that option up. Dustin -- Comment By: Guido van Rossum (gvanrossum) Date: 2001-08-09 11:51 Message: Logged In: YES user_id=6380 Thank you. I've moved this feature request to PEP 42, "Feature Requests". -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=415692&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1628484 ] Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1
Bugs item #1628484, was opened at 2007-01-05 00:45 Message generated for change (Comment added) made by bobatkins You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1628484&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Bob Atkins (bobatkins) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 Initial Comment: This looks like a recurring and somewhat sore topic. For those of us that have been fighting the dreaded: ./Include/pyport.h:730:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." when performing a 64 bit compile. I believe I have identified the problems. All of which are directly related to the Makefile(s) that are generated as part of the configure script. There does not seem to be anything wrong with the configure script or anything else once all of the Makefiles are corrected Python will build 64 bit Although it is possible to pass the following environment variables to configure as is typical on most open source software: CC C compiler command CFLAGS C compiler flags LDFLAGS linker flags, e.g. -L if you have libraries in a nonstandard directory CPPFLAGSC/C++ preprocessor flags, e.g. -I if you have headers in a nonstandard directory CPP C preprocessor These flags are *not* being processed through to the generated Makefiles. This is where the problem is. configure is doing everything right and generating all of the necessary stuff for a 64 bit compile but when the compile is actually performed - the necessary CFLAGS are missing and a 32 bit compile is initiated. Taking a close look at the first failure I found the following: gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -fPIC -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c Where are my CFLAGS??? I ran the configure with: CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ CXXFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ LDFLAGS="-m64 -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ ./configure --prefix=/opt \ --enable-shared \ --libdir=/opt/lib/sparcv9 Checking the config.log and config.status it was clear that the flags were used properly as the configure script ran however, the failure is in the various Makefiles to actually reference the CFLAGS and LDFLAGS. LDFLAGS is simply not included in any of the link stages in the Makefiles and CFLAGS is overidden by BASECFLAGS, OPT and EXTRA_CFLAGS! Ah! EXTRA_CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ make Actually got the core parts to compile for the library and then failed to build the library because - LDFLAGS was missing from the Makefile for the library link stage - :-( Close examination suggests that the OPT environment variable could be used to pass the necessary flags through from conifgure but this still did not help the link stage problems. The fixes are pretty minimal to ensure that the configure variables are passed into the Makefile. My patch to the Makefile.pre.in is attached to this bug report. Once these changes are made Python will build properly for both 32 and 64 bit platforms with the correct CFLAGS and LDFLAGS passed into the configure script. BTW, while this bug is reported under a Solaris/gcc build the patches to Makefile.pre.in should fix similar build issues on all platforms. -- >Comment By: Bob Atkins (bobatkins) Date: 2007-01-07 13:00 Message: Logged In: YES user_id=62 Originator: YES OK, here is the synposis: Run the configure with: $ CFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ CXXFLAGS="-O3 -m64 -mcpu=ultrasparc -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ LDFLAGS="-m64 -L/opt/lib/sparcv9 -R/opt/lib/sparcv9" \ ./configure --prefix=/opt \ --enable-shared \ --libdir=/opt/lib/sparcv9 $ make gcc -pthread -c -fno-strict-aliasing -DNDEBUG-I. -I./Include -fPIC -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c In file included from ./Include/Python.h:57, from ./Modules/python.c:3: ./Include/pyport.h:730:2: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." Cause: Makefile.pre.in does not have substitutions that carry through the CFLAGS that were given to the configure script. In addition although LDFLAGS is carried in from the configure script it is not used in any of the link stages in the Makefile. Minor issue. Makefile.pre.in also does not carry through the 'libdir' configure variable and should be referenci
[ python-Bugs-539444 ] asyncore file wrapper & os.error
Bugs item #539444, was opened at 2002-04-04 12:57 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Hylton (jhylton) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore file wrapper & os.error Initial Comment: The file wrapper makes a file descriptor look like an asycnore socket. When its recv() method is invoked, it calls os.read(). I use this in an application where os.read() occasionally raises os.error (11, 'Resource temporarily unavailable'). I think that asyncore should catch this error and treat it just like EWOULDBLOCK. But I'd like a second opinion. -- >Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 13:35 Message: Logged In: YES user_id=341410 Originator: NO I seem to have misread it as being for send. Presumably they would want to handle EAGAIN/EWOULDBLOCK in recv, though the semantic of returning an empty string when it was polled as being readable, is generally seen as a condition to close the socket. I'm leaning towards closing as invalid, as "fixing" the behavior would result in the semantics of recv being ambiguous. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 11:18 Message: Logged In: YES user_id=21627 Originator: NO Ok; still I wonder what the problem is. In the original report, Jeremy said "should catch this error and treat it just like EWOULDBLOCK". Now, EWOULDBLOCK is handled in dispatcher.connect, dispatcher.accept, and dispatcher.send - not in dispatcher.recv. So what would it help to treat EAGAIN the same way? -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 10:03 Message: Logged In: YES user_id=341410 Originator: NO Jeremy Hylton states what he did to fix it in ZEO. In terms of platform, I would guess that this is likely linux, as multiple people seem to be able to reproduce the error, and you can't reliably use signals in Windows without killing the process. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 02:45 Message: Logged In: YES user_id=21627 Originator: NO Notice that the ZODB issue is marked as fixed. I would like to know how that was fixed, and I still would like to know what operating system this problem occurred on. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-06 22:00 Message: Logged In: YES user_id=341410 Originator: NO I don't see an issue with treating EAGAIN as EWOULDBLOCK. In the cases where EAGAIN != EWOULDBLOCK (in terms of constant value), treating them the same would be the right thing. In the case where the values were the same, nothing would change. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-07 01:03 Message: Logged In: YES user_id=21627 I'm still uncertain what precisely was happening here. What system was this on? On many systems, EAGAIN is EWOULDBLOCK; if that is the case, adding EAGAIN to the places that currently handle EWOULDBLOCK won't change anything. -- Comment By: Jeremy Hylton (jhylton) Date: 2002-04-05 08:44 Message: Logged In: YES user_id=31392 It happens when the file is a pipe. For details, see the ZEO bug report at https://sourceforge.net/tracker/index.php? func=detail&aid=536416&group_id=15628&atid=115628 I've included the traceback from that bug report, too. error: uncaptured python exception, closing channel(exceptions.OSError:[Errno 11] Resource temporarily unavailable [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|poll|92] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|handle_read_event|386] [/home/zope/opt/Python-2.1.2/lib/python2.1/site- packages/ZEO/trigger.py|handle_read|95] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|338] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|recv|520]) Exception exceptions.OSError: (9, 'Bad file descriptor') in ignored -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 01:00 Message: Logged In: YES user_id=21627 Can you report details of the file that returns EWOULDBLOCK? This is not s
[ python-Bugs-539444 ] asyncore file wrapper & os.error
Bugs item #539444, was opened at 2002-04-04 22:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Hylton (jhylton) Assigned to: Josiah Carlson (josiahcarlson) Summary: asyncore file wrapper & os.error Initial Comment: The file wrapper makes a file descriptor look like an asycnore socket. When its recv() method is invoked, it calls os.read(). I use this in an application where os.read() occasionally raises os.error (11, 'Resource temporarily unavailable'). I think that asyncore should catch this error and treat it just like EWOULDBLOCK. But I'd like a second opinion. -- >Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 22:49 Message: Logged In: YES user_id=21627 Originator: NO What still puzzles me is why recv is invoked at all. According to the traceback, it was invoked because poll() indicated a read event for the pipe, yet trying to read from it failed with EAGAIN. Either there still is a bug in asyncore, or there is a bug in the operating system, or the traceback is bogus. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 22:35 Message: Logged In: YES user_id=341410 Originator: NO I seem to have misread it as being for send. Presumably they would want to handle EAGAIN/EWOULDBLOCK in recv, though the semantic of returning an empty string when it was polled as being readable, is generally seen as a condition to close the socket. I'm leaning towards closing as invalid, as "fixing" the behavior would result in the semantics of recv being ambiguous. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 20:18 Message: Logged In: YES user_id=21627 Originator: NO Ok; still I wonder what the problem is. In the original report, Jeremy said "should catch this error and treat it just like EWOULDBLOCK". Now, EWOULDBLOCK is handled in dispatcher.connect, dispatcher.accept, and dispatcher.send - not in dispatcher.recv. So what would it help to treat EAGAIN the same way? -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 19:03 Message: Logged In: YES user_id=341410 Originator: NO Jeremy Hylton states what he did to fix it in ZEO. In terms of platform, I would guess that this is likely linux, as multiple people seem to be able to reproduce the error, and you can't reliably use signals in Windows without killing the process. -- Comment By: Martin v. Löwis (loewis) Date: 2007-01-07 11:45 Message: Logged In: YES user_id=21627 Originator: NO Notice that the ZODB issue is marked as fixed. I would like to know how that was fixed, and I still would like to know what operating system this problem occurred on. -- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-01-07 07:00 Message: Logged In: YES user_id=341410 Originator: NO I don't see an issue with treating EAGAIN as EWOULDBLOCK. In the cases where EAGAIN != EWOULDBLOCK (in terms of constant value), treating them the same would be the right thing. In the case where the values were the same, nothing would change. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-07 11:03 Message: Logged In: YES user_id=21627 I'm still uncertain what precisely was happening here. What system was this on? On many systems, EAGAIN is EWOULDBLOCK; if that is the case, adding EAGAIN to the places that currently handle EWOULDBLOCK won't change anything. -- Comment By: Jeremy Hylton (jhylton) Date: 2002-04-05 18:44 Message: Logged In: YES user_id=31392 It happens when the file is a pipe. For details, see the ZEO bug report at https://sourceforge.net/tracker/index.php? func=detail&aid=536416&group_id=15628&atid=115628 I've included the traceback from that bug report, too. error: uncaptured python exception, closing channel(exceptions.OSError:[Errno 11] Resource temporarily unavailable [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|poll|92] [/home/zope/opt/Python- 2.1.2/lib/python2.1/asyncore.py|handle_read_event|386] [/home/zope/opt/Python-2.1.2/lib/python2.1/site- packages/ZEO/trigger.py|handle_read|95] [/home/zope/opt/Python-
[ python-Bugs-1569622 ] Backward incompatibility in logging.py
Bugs item #1569622, was opened at 2006-10-02 16:10 Message generated for change (Comment added) made by mklaas You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569622&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Closed Resolution: Fixed >Priority: 9 Private: No Submitted By: Mike Klaas (mklaas) >Assigned to: Neal Norwitz (nnorwitz) >Summary: Backward incompatibility in logging.py Initial Comment: LogRecord.__init__ changed in a backward incompatible way in python 2.5 (added one parameter). There is no mention of this breakage in the release notes, nor has the documentation of the module been updated (http://docs.python.org/lib/node424.html) -- >Comment By: Mike Klaas (mklaas) Date: 2007-01-07 18:19 Message: Logged In: YES user_id=1611720 Originator: YES This fix should be back-ported to 2.5 maint: r52100,52101,52102 perhaps also r52555,52556? -- Comment By: Vinay Sajip (vsajip) Date: 2006-10-03 11:22 Message: Logged In: YES user_id=308438 Documentation now updated in CVS. Also changed the added "func" parameter to have a default value of None. Sorry for the inconvenience caused. -- Comment By: Mike Klaas (mklaas) Date: 2006-10-03 10:14 Message: Logged In: YES user_id=1611720 It is incompatible as code written for 2.4 will break in 2.5, and vice-versa (this is a required parameter, not an optional parameter, and the change could have been made in a backward-compatible way). You're right that the documentation fix is the important thing. -- Comment By: Georg Brandl (gbrandl) Date: 2006-10-02 23:11 Message: Logged In: YES user_id=849994 I don't see why adding one parameter is backwards incompatible, but it's true that the docs must be updated. Assigning to Vinay. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569622&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1569622 ] Backward incompatibility in logging.py
Bugs item #1569622, was opened at 2006-10-02 16:10 Message generated for change (Settings changed) made by mklaas You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569622&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 >Status: Open Resolution: Fixed Priority: 9 Private: No Submitted By: Mike Klaas (mklaas) Assigned to: Neal Norwitz (nnorwitz) Summary: Backward incompatibility in logging.py Initial Comment: LogRecord.__init__ changed in a backward incompatible way in python 2.5 (added one parameter). There is no mention of this breakage in the release notes, nor has the documentation of the module been updated (http://docs.python.org/lib/node424.html) -- Comment By: Mike Klaas (mklaas) Date: 2007-01-07 18:19 Message: Logged In: YES user_id=1611720 Originator: YES This fix should be back-ported to 2.5 maint: r52100,52101,52102 perhaps also r52555,52556? -- Comment By: Vinay Sajip (vsajip) Date: 2006-10-03 11:22 Message: Logged In: YES user_id=308438 Documentation now updated in CVS. Also changed the added "func" parameter to have a default value of None. Sorry for the inconvenience caused. -- Comment By: Mike Klaas (mklaas) Date: 2006-10-03 10:14 Message: Logged In: YES user_id=1611720 It is incompatible as code written for 2.4 will break in 2.5, and vice-versa (this is a required parameter, not an optional parameter, and the change could have been made in a backward-compatible way). You're right that the documentation fix is the important thing. -- Comment By: Georg Brandl (gbrandl) Date: 2006-10-02 23:11 Message: Logged In: YES user_id=849994 I don't see why adding one parameter is backwards incompatible, but it's true that the docs must be updated. Assigning to Vinay. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569622&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com