[ python-Feature Requests-1634717 ] csv.DictWriter: Include offending name in error message
Feature Requests item #1634717, was opened at 2007-01-13 11:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1634717&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Gabriel Genellina (gagenellina) Assigned to: Nobody/Anonymous (nobody) Summary: csv.DictWriter: Include offending name in error message Initial Comment: In csv.py, class DictWriter, method _dict_to_list, when rowdict contains a key that is not a known field name, a ValueError is raised, but no reference to the offending name is given. As the code iterates along the dict keys, and stops at the first unknown one, its trivial to include such information. Replace lines: if k not in self.fieldnames: raise ValueError, "dict contains fields not in fieldnames" with: if k not in self.fieldnames: raise ValueError, "dict contains field not in fieldnames: %r" % k -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1634717&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES user_id=310088 Originator: YES File Added: Makefile -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.py -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES user_id=310088 Originator: YES File Added: Makefile -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&grou
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:50 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.output -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.py -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES user_
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:51 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.output2 -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:50 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.output -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:49 Message: Logged In: YES
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:52 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.py.output -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:51 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.output2 -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:50 Message: Logged In: YE
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:52 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.py.output2 -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:52 Message: Logged In: YES user_id=310088 Originator: YES File Added: extract_frame.py.output -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 15:51 Message: Logged In:
[ python-Feature Requests-1634770 ] Please provide rsync-method in the urllib[2] module
Feature Requests item #1634770, was opened at 2007-01-13 18:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1634770&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matthias Klose (doko) Assigned to: Nobody/Anonymous (nobody) Summary: Please provide rsync-method in the urllib[2] module Initial Comment: [forwarded from http://bugs.debian.org/323213] sometimes it would be nice to be able to just open rsync-connections directly via the urllib-methode (like http- and ftp-ressources). -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1634770&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634774 ] locale 1251 does not convert to upper case properly
Bugs item #1634774, was opened at 2007-01-13 18:30
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Dobrokotov (dobrokot)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale 1251 does not convert to upper case properly
Initial Comment:
# -*- coding: 1251 -*-
import locale
locale.setlocale(locale.LC_ALL, ".1251") #locale name may be Windows specific?
#---
print chr(184), chr(168)
assert chr(255).upper() == chr(223) #OK
assert chr(184).upper() == chr(168) #fail
#---
assert 'q'.upper() == 'Q' #OK
assert 'ж'.upper() == 'Ж' #OK
assert 'Ж'.upper() == 'Ж' #OK
assert 'я'.upper() == 'Я' #OK
assert u'ё'.upper() == u'Ё' #OK (locale independent)
assert 'ё'.upper() == 'Ё' #fail
I suppose incorrect realization of uppercase like
if ('a' <= c && c <= 'я')
return c+'Я'-'я'
symbol 'ё' (184 in cp1251) is not in range 'a'-'я'
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634774 ] locale 1251 does not convert to upper case properly
Bugs item #1634774, was opened at 2007-01-13 18:30
Message generated for change (Comment added) made by dobrokot
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Dobrokotov (dobrokot)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale 1251 does not convert to upper case properly
Initial Comment:
# -*- coding: 1251 -*-
import locale
locale.setlocale(locale.LC_ALL, ".1251") #locale name may be Windows specific?
#---
print chr(184), chr(168)
assert chr(255).upper() == chr(223) #OK
assert chr(184).upper() == chr(168) #fail
#---
assert 'q'.upper() == 'Q' #OK
assert 'ж'.upper() == 'Ж' #OK
assert 'Ж'.upper() == 'Ж' #OK
assert 'я'.upper() == 'Я' #OK
assert u'ё'.upper() == u'Ё' #OK (locale independent)
assert 'ё'.upper() == 'Ё' #fail
I suppose incorrect realization of uppercase like
if ('a' <= c && c <= 'я')
return c+'Я'-'я'
symbol 'ё' (184 in cp1251) is not in range 'a'-'я'
--
>Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 18:49
Message:
Logged In: YES
user_id=1538986
Originator: YES
C-CRT library fucntion toupper('ё') works properly, if I set
setlocale(LC_ALL, ".1251")
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634774 ] locale 1251 does not convert to upper case properly
Bugs item #1634774, was opened at 2007-01-13 18:30
Message generated for change (Comment added) made by dobrokot
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Dobrokotov (dobrokot)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale 1251 does not convert to upper case properly
Initial Comment:
# -*- coding: 1251 -*-
import locale
locale.setlocale(locale.LC_ALL, ".1251") #locale name may be Windows specific?
#---
print chr(184), chr(168)
assert chr(255).upper() == chr(223) #OK
assert chr(184).upper() == chr(168) #fail
#---
assert 'q'.upper() == 'Q' #OK
assert 'ж'.upper() == 'Ж' #OK
assert 'Ж'.upper() == 'Ж' #OK
assert 'я'.upper() == 'Я' #OK
assert u'ё'.upper() == u'Ё' #OK (locale independent)
assert 'ё'.upper() == 'Ё' #fail
I suppose incorrect realization of uppercase like
if ('a' <= c && c <= 'я')
return c+'Я'-'я'
symbol 'ё' (184 in cp1251) is not in range 'a'-'я'
--
>Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 18:51
Message:
Logged In: YES
user_id=1538986
Originator: YES
sorry, I mean
toupper((int)(unsigned char)'ё')
not just toupper('ё')
--
Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 18:49
Message:
Logged In: YES
user_id=1538986
Originator: YES
C-CRT library fucntion toupper('ё') works properly, if I set
setlocale(LC_ALL, ".1251")
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1633630 ] class derived from float evaporates under +=
Bugs item #1633630, was opened at 2007-01-11 23:49 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633630&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Type/class unification Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matthias Klose (doko) Assigned to: Nobody/Anonymous (nobody) Summary: class derived from float evaporates under += Initial Comment: [forwarded from http://bugs.debian.org/345373] There seems to be a bug in classes derived from float. For instance, consider the following: >>> class Float(float): ... def __init__(self, v): ... float.__init__(self, v) ... self.x = 1 ... >>> a = Float(2.0) >>> b = Float(3.0) >>> type(a) >>> type(b) >>> a += b >>> type(a) Now, the type of a has silently changed. It was a Float, a derived class with all kinds of properties, and it became a float -- a plain vanilla number. My understanding is that this is incorrect, and certainly unexpected. If it *is* correct, it certainly deserves mention somewhere in the documentation. It seems that Float.__iadd__(a, b) should be called. This defaults to float.__iadd__(a, b), which should increment the float part of the object while leaving the rest intact. A possible explanation for this problem is that float.__iadd__ is not actually defined, and so it falls through to a = float.__add__(a, b), which assigns a float to a. This interpretation seems to be correct, as one can add a destructor to the Float class: >>> class FloatD(float): ... def __init__(self, v): ... float.__init__(self, v) ... self.x = 1 ... def __del__(self): ... print 'Deleting FloatD class, losing x=', self.x ... >>> a = FloatD(2.0) >>> b = FloatD(3.0) >>> a += b Deleting FloatD class, losing x= 1 >>> -- >Comment By: Georg Brandl (gbrandl) Date: 2007-01-13 17:57 Message: Logged In: YES user_id=849994 Originator: NO You don't need augmented assign for that, just doing "a+b" will give you a float too. -- Comment By: Jim Jewett (jimjjewett) Date: 2007-01-12 21:26 Message: Logged In: YES user_id=764593 Originator: NO Python float objects are immutable and can be shared. Therefore, their values cannot be modified -- which is why it falls back to not-in-place assignment. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633630&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1599254 ] mailbox: other programs' messages can vanish without trace
Bugs item #1599254, was opened at 2006-11-19 16:03 Message generated for change (Comment added) made by baikie You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 9 Private: No Submitted By: David Watson (baikie) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox: other programs' messages can vanish without trace Initial Comment: The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion. I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so). A possible fix is attached. Instead of new_file being renamed, its contents are copied back to the original file. If file.truncate() is available, the mailbox is then truncated to size. Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+. In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition. Any alternative ideas? Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set. -- >Comment By: David Watson (baikie) Date: 2007-01-13 18:32 Message: Logged In: YES user_id=1504904 Originator: YES I like the warning idea - it seems appropriate if the problem is relatively rare. How about this? File Added: mailbox-fcntl-warn.diff -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-12 19:41 Message: Logged In: YES user_id=11375 Originator: NO One OS/2 port lacks truncate(), and so does RISCOS. -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-12 18:41 Message: Logged In: YES user_id=11375 Originator: NO I realized that making flush() invalidate keys breaks the final example in the docs, which loops over inbox.iterkeys() and removes messages, doing a pack() after each message. Which platforms lack file.truncate()? Windows has it; POSIX has it, so modern Unix variants should all have it. Maybe mailbox should simply raise an exception (or trigger a warning?) if truncate is missing, and we should then assume that flush() has no effect upon keys. -- Comment By: A.M. Kuchling (akuchling) Date: 2007-01-12 17:12 Message: Logged In: YES user_id=11375 Originator: NO So shall we document flush() as invalidating keys, then? -- Comment By: David Watson (baikie) Date: 2007-01-06 19:57 Message: Logged In: YES user_id=1504904 Originator: YES Oops, length checking had made the first two lines of this patch redundant; update-toc applies OK with fuzz. File Added: mailbox-copy-back-new.diff -- Comment By: David Watson (baikie) Date: 2007-01-06 15:30 Message: Logged In: YES user_id=1504904 Originator: YES File Added: mailbox-copy-back-53287.diff -- Comment By: David Watson (baikie) Date: 2007-01-06 15:24 Message: Logged In: YES user_id=1504904 Originator: YES Aack, yes, that should be _next_user_key. Attaching a fixed version. I've been thinking, though: flush() does in fact invalidate the keys on platforms without a file.truncate(), when the fcntl() lock is momentarily released afterwards. It seems hard to avoid this as, perversely, fcntl() locks are supposed to be released automatically on all file descriptors referring to the file whenever the process closes any one of them - even one the lock was never set on. So, code using mailbox.py such platforms could inadvertently be carrying keys across an unlocked period, which is not made safe by the update-toc patch (as it's only meant to avert disasters resulting from doing this *and* rebuilding the table of contents, *assuming* that another process hasn't deleted or rearranged messages). File Added: mailbox-updat
[ python-Bugs-1313119 ] urlparse "caches" parses regardless of encoding
Bugs item #1313119, was opened at 2005-10-04 19:57
Message generated for change (Comment added) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1313119&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ken Kinder (kkinder)
>Assigned to: Nobody/Anonymous (nobody)
Summary: urlparse "caches" parses regardless of encoding
Initial Comment:
The issue can be summarized with this code:
>>> urlparse.urlparse(u'http://www.python.org/doc')
(u'http', u'www.python.org', u'/doc', '', '', '')
>>> urlparse.urlparse('http://www.python.org/doc')
(u'http', u'www.python.org', u'/doc', '', '', '')
Once the urlparse library has "cached" a URL, it stores
the resulting value of that cache regardless of
datatype. Notice that in the second use of urlparse, I
passed it a STRING and got back a UNICODE object.
This can be quite confusing when, as a developer, you
think you've already encoded all your objects, you use
urlparse, and all of a sudden you have unicode objects
again, when you expected to have strings.
--
>Comment By: M.-A. Lemburg (lemburg)
Date: 2007-01-13 20:25
Message:
Logged In: YES
user_id=38388
Originator: NO
Unassigning: I don't use urlparse, so can't comment.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1313119&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-967986 ] file.encoding doesn't apply to file.write
Bugs item #967986, was opened at 2004-06-07 09:00 Message generated for change (Comment added) made by lemburg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=967986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Unicode Group: Python 2.3 >Status: Closed >Resolution: Wont Fix Priority: 5 Private: No Submitted By: Matthew Mueller (donut) Assigned to: M.-A. Lemburg (lemburg) Summary: file.encoding doesn't apply to file.write Initial Comment: In python2.3 printing unicode to an appropriate terminal actually works. But using sys.stdout.write doesn't. Ex: Python 2.3.4 (#2, May 29 2004, 03:31:27) [GCC 3.3.3 (Debian 20040417)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> sys.stdout.encoding 'UTF-8' >>> u=u'\u3053\u3093\u306b\u3061\u308f' >>> print u こんにちわ >>> sys.stdout.write(u) Traceback (most recent call last): File "", line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-4: ordinal not in range(128) The file object docs say: "encoding The encoding that this file uses. When Unicode strings are written to a file, they will be converted to byte strings using this encoding. ..." Which indicates to me that it is supposed to work. -- >Comment By: M.-A. Lemburg (lemburg) Date: 2007-01-13 20:26 Message: Logged In: YES user_id=38388 Originator: NO Not sure whether this is still the case. No patches were provided, so closing the feature request. -- Comment By: M.-A. Lemburg (lemburg) Date: 2004-07-23 12:23 Message: Logged In: YES user_id=38388 The encoding feature is currently only implemented for printing. We could also add it to .write() and .writelines() ... patches are welcome. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=967986&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-928297 ] platform.libc_ver() fails on Cygwin
Bugs item #928297, was opened at 2004-04-02 16:55
Message generated for change (Comment added) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=928297&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: George Yoshida (quiver)
Assigned to: M.-A. Lemburg (lemburg)
Summary: platform.libc_ver() fails on Cygwin
Initial Comment:
>>> import platform
>>> platform.libc_ver()
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/lib/python2.3/platform.py", line 134, in libc_ver
f = open(executable,'rb')
IOError: [Errno 2] No such file or
directory: '/usr/bin/python'
The problem is that on Cygwin sys.executable
returns /path/to/python, but since Cygwin is running on
Windows, sys.executable is a symbolic link
to /path/to/python.exe.
>>> import os, sys
>>> os.path.exists(sys.executable)
True
>>> os.path.isfile(sys.executable)
True
>>> file(sys.executable)
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or
directory: '/usr/bin/python'
>>> os.path.islink(sys.executable)
True
>>> os.path.realpath(sys.executable)
'/usr/bin/python2.3.exe'
>>> file(sys.executable + '.exe')
Following is the info about the machine I tested:
>>> from platform import *
>>> platform()
'CYGWIN_NT-5.0-1.5.7-0.109-3-2-i686-32bit'
>>> python_compiler()
'GCC 3.3.1 (cygming special)'
>>> python_build()
(1, 'Dec 30 2003 08:29:25')
>>> python_version()
'2.3.3'
>>> uname()
('CYGWIN_NT-5.0', 'my_user_name', '1.5.7
(0.109/3/2)', '2004-01-30 19:32', 'i686', '')
--
>Comment By: M.-A. Lemburg (lemburg)
Date: 2007-01-13 20:29
Message:
Logged In: YES
user_id=38388
Originator: NO
Since Cygwin doesn't appear to use the GLibC, there's no surprise in
libc_ver() not returning any useful information.
--
Comment By: George Yoshida (quiver)
Date: 2004-07-30 01:29
Message:
Logged In: YES
user_id=671362
Sorry for my late response, Marc.
> Would applying os.path.realpath() to sys.executable before
> trying to open that file fix the problem on Cygwin ?
That change fixes the IO problem. After this, it doesn't raise
IOError. The result of platform.libc_ver() is as follows:
>>> import platform
>>> platform.libc_ver()
('', '')
> Another question: does using libc_ver() even make sense on
> cygwin ?
As far as I have checked, it doesn't look like so. According to
the Cygwin FAQ[*], Cygwin doesn't use glibc, although it
says that there's a counterpart(called ``newlib'') in Cygwin. C
runtime embedded into cygwin1.dll uses newlib.
Experienced C & Cygwin programmers might anser this
question more precisely.
[*] Where is glibc? :
http://rustam.uwp.edu/support/faq.html#SEC88
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-07-23 12:34
Message:
Logged In: YES
user_id=38388
Would applying os.path.realpath() to sys.executable before
trying to open that file fix the problem on Cygwin ?
Another question: does using libc_ver() even make sense on
cygwin ?
libc_ver() was never intended to be used on non-*nix
platforms. I don't even know whether it works on other
platforms than Linux.
--
Comment By: George Yoshida (quiver)
Date: 2004-04-03 06:20
Message:
Logged In: YES
user_id=671362
First, I need to correct my previous post. 'symbolic' was
unrelated.
Python on Cygwin does't like exe files that doesn't end
with '.exe'.
I think changing fileobject.c to support I/O exe files on
Cygwin whether it ends with '.exe' or not is the way to go.
Is there anyone who can do that? It's beyoond my skill level.
$ ls -l /usr/bin/python*
lrwxrwxrwx 1 abel Users24 Jan 1 01:34 /usr/bin/python -
> python2.3.exe
lrwxrwxrwx 1 abel Users24 Jan 1
01:34 /usr/bin/python.exe -> python2.3.exe
-rwxrwxrwx 1 abel Users 4608 Dec 30
22:32 /usr/bin/python2.3.exe
>>> file('/usr/bin/python')
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or directory: '/usr/bin/python'
>>> file('/usr/bin/python.exe')
>>> file('/usr/bin/python2.3')
Traceback (most recent call last):
File "", line 1, in ?
IOError: [Errno 2] No such file or
directory: '/usr/bin/python2.3'
>>> file('/usr/bin/python2.3.exe')
--
Comment By: M.-A. Lemburg (lemburg)
Date: 2004-04-02 17:59
Message:
Logged In: YES
user_id=38388
Patches are welcome :-)
I don't have cygwin installed, so there's nothing much
I can do about this.
[ python-Bugs-1634774 ] locale 1251 does not convert to upper case properly
Bugs item #1634774, was opened at 2007-01-13 18:30
Message generated for change (Comment added) made by dobrokot
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Dobrokotov (dobrokot)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale 1251 does not convert to upper case properly
Initial Comment:
# -*- coding: 1251 -*-
import locale
locale.setlocale(locale.LC_ALL, ".1251") #locale name may be Windows specific?
#---
print chr(184), chr(168)
assert chr(255).upper() == chr(223) #OK
assert chr(184).upper() == chr(168) #fail
#---
assert 'q'.upper() == 'Q' #OK
assert 'ж'.upper() == 'Ж' #OK
assert 'Ж'.upper() == 'Ж' #OK
assert 'я'.upper() == 'Я' #OK
assert u'ё'.upper() == u'Ё' #OK (locale independent)
assert 'ё'.upper() == 'Ё' #fail
I suppose incorrect realization of uppercase like
if ('a' <= c && c <= 'я')
return c+'Я'-'я'
symbol 'ё' (184 in cp1251) is not in range 'a'-'я'
--
>Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 22:08
Message:
Logged In: YES
user_id=1538986
Originator: YES
forgot to mention used python version -
http://www.python.org/ftp/python/2.5/python-2.5.msi
--
Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 18:51
Message:
Logged In: YES
user_id=1538986
Originator: YES
sorry, I mean
toupper((int)(unsigned char)'ё')
not just toupper('ё')
--
Comment By: Ivan Dobrokotov (dobrokot)
Date: 2007-01-13 18:49
Message:
Logged In: YES
user_id=1538986
Originator: YES
C-CRT library fucntion toupper('ё') works properly, if I set
setlocale(LC_ALL, ".1251")
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634774&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 16:46 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-13 23:14 Message: Logged In: YES user_id=344921 Originator: NO The first thing to check is if the subprocesses have different sets up file descriptors when you launch them from Python and C, respectively. On Linux, do /proc/$thepid/fd in both cases and compare the output. Does it matter if you use close_fds=1? -- Comment By: Florent Rougon (frougon) Date: 2007-01-13 16:52 Message: Logged In: YES user_id=31
[ python-Bugs-1634343 ] subprocess swallows empty arguments under win32
Bugs item #1634343, was opened at 2007-01-12 21:40
Message generated for change (Comment added) made by astrand
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634343&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Patrick Mézard (pmezard)
>Assigned to: Peter Åstrand (astrand)
Summary: subprocess swallows empty arguments under win32
Initial Comment:
Hello,
empty arguments are not quoted by subprocess.list2cmdline. Therefore nothing is
concatenated with other arguments. To reproduce it:
test-empty.py
"""
import sys
print sys.argv
"""
then:
"""
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> p = subprocess.Popen(['python', 'test-empty.py', ''],
>>> stdout=subprocess.PIPE)
>>> p.communicate()
("['test-empty.py']\r\n", None)
"""
To solve it:
"""
--- a\subprocess.py 2007-01-12 21:38:57.734375000 +0100
+++ b\subprocess.py 2007-01-12 21:34:08.40625 +0100
@@ -499,7 +499,7 @@
if result:
result.append(' ')
-needquote = (" " in arg) or ("\t" in arg)
+needquote = (" " in arg) or ("\t" in arg) or not arg
if needquote:
result.append('"')
"""
Regard,
Patrick Mézard
--
>Comment By: Peter Åstrand (astrand)
Date: 2007-01-13 23:37
Message:
Logged In: YES
user_id=344921
Originator: NO
Fixed in revision 53412 (trunk) and 53413 (25-maint).
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634343&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1590864 ] import deadlocks when using PyObjC threads
Bugs item #1590864, was opened at 2006-11-05 17:06 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Tsai (michaeltsai) >Assigned to: Nobody/Anonymous (nobody) >Summary: import deadlocks when using PyObjC threads Initial Comment: When I use subprocess.py from a child thread, sometimes it deadlocks. I determined that the new process is blocked during an import: #0 0x90024427 in semaphore_wait_signal_trap () #1 0x90028414 in pthread_cond_wait () #2 0x004c77bf in PyThread_acquire_lock (lock=0x3189a0, waitflag=1) at Python/thread_pthread.h:452 #3 0x004ae2a6 in lock_import () at Python/import.c:266 #4 0x004b24be in PyImport_ImportModuleLevel (name=0xaad74 "errno", globals=0xbaed0, locals=0x502aa0, fromlist=0xc1378, level=-1) at Python/import.c:2054 #5 0x0048d2e2 in builtin___import__ (self=0x0, args=0x53724c90, kwds=0x0) at Python/bltinmodule.c:47 #6 0x0040decb in PyObject_Call (func=0xa94b8, arg=0x53724c90, kw=0x0) at Objects/abstract.c:1860 and that the code in question is in os.py: def _execvpe(file, args, env=None): from errno import ENOENT, ENOTDIR I think the problem is that since exec (the C function) hasn't yet been called in the new process, it's inherited from the fork a lock that's already held. The main process will eventually release its copy of the lock, but this will not unlock it in the new process, so it deadlocks. If I change os.py so that it imports the constants outside of _execvpe, the new process no longer blocks in this way. This is on Mac OS X 10.4.8. -- >Comment By: Peter Åstrand (astrand) Date: 2007-01-13 23:42 Message: Logged In: YES user_id=344921 Originator: NO Since both the reporter and I believes that this is not a bug in the subprocess module, I'm stepping back. -- Comment By: Michael Tsai (michaeltsai) Date: 2007-01-07 18:09 Message: Logged In: YES user_id=817528 Originator: YES I don't have time at the moment to write sample code that reproduces this. But, FYI, I was using PyObjC to create the threads. It might not happen with "threading" threads. And second, I think it's a bug in os.py, not in subprocess.py. Sorry for the confusion. -- Comment By: Peter Åstrand (astrand) Date: 2007-01-07 15:10 Message: Logged In: YES user_id=344921 Originator: NO Can you provide a test case or sample code that demonstrates this problem? I'm a bit unsure of if this really is a subprocess bug or a more general Python bug. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1590864&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1634739 ] Problem running a subprocess
Bugs item #1634739, was opened at 2007-01-13 15:46 Message generated for change (Comment added) made by frougon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1634739&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Florent Rougon (frougon) Assigned to: Nobody/Anonymous (nobody) Summary: Problem running a subprocess Initial Comment: Hello, I have a problem running a subprocess from Python (see below). I first ran into it with the subprocess module, but it's also triggered by a simple os.fork() followed by os.execvp(). So, what is the problem, exactly? I have written the exact same minimal program in C and in Python, which uses fork() and execvp() in the most straightforward way to run the following command: transcode -i /tmp/file.mpg -c 100-101 -o snapshot -y im,null -F png (whose effect is to extract the 100th frame of /tmp/file.mpg and store it into snapshot.png) The C program runs fast with no error, while the one in Python takes from 60 to 145 times longer (!), and triggers error messages from transcode. This shouldn't happen, since both programs are merely calling transcode in the same way to perform the exact same thing. Experiments 1. First, I run the C program (extract_frame) twice on a first .mpg file (MPEG 2 PS) [the first time fills the block IO cache], and store the output in extract_frame.output: % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.82s user 0.33s system 53% cpu 2.175 total % time ./extract_frame >extract_frame.output 2>&1 ./extract_frame > extract_frame.output 2>& 1 0.79s user 0.29s system 96% cpu 1.118 total Basically, this takes 1 or 2 seconds. extract_frame.output is attached. Second, I run the Python program (extract_frame.py) on the same .mpg file, and store the output in extract_frame.py.output: % time ./extract_frame.py >extract_frame.py.output 2>&1 ./extract_frame.py > extract_frame.py.output 2>& 1 81.59s user 25.98s system 66% cpu 2:42.51 total This takes more than 2 *minutes*, not seconds! (of course, the system is idle for all tests) In extract_frame.py.output, the following error message appears quickly after the process is started: failed to write Y plane of frame(demuxer.c) write program stream packet: Broken pipe which is in fact composed of two error messages, the second one starting at "(demuxer.c)". Once these messages are printed, the transcode subprocesses[1] seem to hang (with relatively high CPU usage), but eventually complete, after 2 minutes or so. There are no such error messages in extract_frame.output. 2. Same test with another .mpg file. As far as time is concerned, we have the same problem: [C program] % time ./extract_frame >extract_frame.output2 2>&1 ./extract_frame > extract_frame.output2 2>& 1 0.73s user 0.28s system 43% cpu 2.311 total [Python program] % time ./extract_frame.py >extract_frame.py.output2 2>&1 ./extract_frame.py > extract_frame.py.output2 2>& 1 92.84s user 12.20s system 76% cpu 2:18.14 total We also get the first error message in extract_frame.py.output2: failed to write Y plane of frame when running extract_frame.py, but this time, we do *not* have the second error message: (demuxer.c) write program stream packet: Broken pipe All this is reproducible with Python 2.3, 2.4 and 2.5 (Debian packages in sarge for 2.3 and 2.4, vanilla Python 2.5). % python2.5 -c 'import sys; print sys.version; print "%x" % sys.hexversion' 2.5 (r25:51908, Jan 5 2007, 17:35:09) [GCC 3.3.5 (Debian 1:3.3.5-13)] 20500f0 % transcode --version transcode v1.0.2 (C) 2001-2003 Thomas Oestreich, 2003-2004 T. Bitterberg I'd hazard that Python is tweaking some process or threading parameter that is inherited by subprocesses and disturbs transcode, which doesn't happen when calling fork() and execvp() from a C program, but am unfortunately unable to precisely diagnose the problem. Many thanks for considering. Regards, Florent [1] Plural because the transcode process spawns several childs: tcextract, tcdemux, etc. -- >Comment By: Florent Rougon (frougon) Date: 2007-01-13 23:37 Message: Logged In: YES user_id=310088 Originator: YES Hi Peter, At the very beginning, it seems the fds are the same in the child processes running transcode in each implementation (C, Python). With the C version, I got: total 5 dr-x-- 2 flo users 0 2007-01-14 00:12 . dr-xr-xr-x 4 flo users 0 2007-01-14 00:12 .. lrwx-- 1 flo users 64 2007-01-14 00:12 0 -> /dev/pts/0 l-wx-- 1 flo users 64 2007-01-14 00:12 1 -> /home/flo/tmp/transcode-test/extract_
[ python-Bugs-1619659 ] htonl, ntohl don't handle negative longs
Bugs item #1619659, was opened at 2006-12-20 13:42 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1619659&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: htonl, ntohl don't handle negative longs Initial Comment: >>> htonl(-5) -67108865 >>> htonl(-5L) Traceback (most recent call last): File "", line 1, in ? OverflowError: can't convert negative value to unsigned long It works fine in 2.1 and 2.2, but fails in 2.3, 2.4, 2.5. htons, ntohs do not appear to have the bug, but I'm not 100% sure. -- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-01-13 19:08 Message: Logged In: YES user_id=6380 Originator: NO mark-roberts, where's your patch? -- Comment By: Mark Roberts (mark-roberts) Date: 2006-12-29 21:15 Message: Logged In: YES user_id=1591633 Originator: NO Hmmm, yes, I see a problem. At the very least, I think we may be wanting some consistency between the acceptance of ints and longs. Also, I think we should return an unsigned long instead of just a long (which can be negative). I've got a patch right now to make htonl, ntohl, htons, and ntohs never return a negative number. I'm rather waffling to the idea of whether we should accept negative numbers at all in any of the functions. The behavior is undefined, and it is, afterall, better not to guess what a user intended. However, consistency should be a desirable goal, and we should accept make the interface consistent for both ints and longs. Mark -- Comment By: Adam Olsen (rhamphoryncus) Date: 2006-12-28 16:37 Message: Logged In: YES user_id=12364 Originator: YES I forgot to mention it, but the only reason htonl should get passed a negative number is that it (and possibly struct?) produce a negative number. Changing them to always produce positive numbers may be an alternative solution. Or we may want to do both, always producing positive while also accepting negative. -- Comment By: Mark Roberts (mark-roberts) Date: 2006-12-26 04:24 Message: Logged In: YES user_id=1591633 Originator: NO >From man page for htonl and friends: #include uint32_t htonl(uint32_t hostlong); uint16_t htons(uint16_t hostshort); uint32_t ntohl(uint32_t netlong); uint16_t ntohs(uint16_t netshort); Python does call these underlying functions in Modules/socketmodule.c. The problem comes from that PyLong_AsUnsignedLong() called in socket_htonl() specifically checks to see that the value cannot be less than 0. The error checking was rather exquisite, I might add. - Mark -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1619659&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1619659 ] htonl, ntohl don't handle negative longs
Bugs item #1619659, was opened at 2006-12-20 12:42 Message generated for change (Comment added) made by mark-roberts You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1619659&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: htonl, ntohl don't handle negative longs Initial Comment: >>> htonl(-5) -67108865 >>> htonl(-5L) Traceback (most recent call last): File "", line 1, in ? OverflowError: can't convert negative value to unsigned long It works fine in 2.1 and 2.2, but fails in 2.3, 2.4, 2.5. htons, ntohs do not appear to have the bug, but I'm not 100% sure. -- Comment By: Mark Roberts (mark-roberts) Date: 2007-01-14 01:36 Message: Logged In: YES user_id=1591633 Originator: NO It is here: https://sourceforge.net/tracker/index.php?func=detail&aid=1635058&group_id=5470&atid=305470 I apologize for not getting to this sooner, but I've been working like a frenzied devil at work. Things have been really hectic with our customers wanting year end reports. -- Comment By: Guido van Rossum (gvanrossum) Date: 2007-01-13 18:08 Message: Logged In: YES user_id=6380 Originator: NO mark-roberts, where's your patch? -- Comment By: Mark Roberts (mark-roberts) Date: 2006-12-29 20:15 Message: Logged In: YES user_id=1591633 Originator: NO Hmmm, yes, I see a problem. At the very least, I think we may be wanting some consistency between the acceptance of ints and longs. Also, I think we should return an unsigned long instead of just a long (which can be negative). I've got a patch right now to make htonl, ntohl, htons, and ntohs never return a negative number. I'm rather waffling to the idea of whether we should accept negative numbers at all in any of the functions. The behavior is undefined, and it is, afterall, better not to guess what a user intended. However, consistency should be a desirable goal, and we should accept make the interface consistent for both ints and longs. Mark -- Comment By: Adam Olsen (rhamphoryncus) Date: 2006-12-28 15:37 Message: Logged In: YES user_id=12364 Originator: YES I forgot to mention it, but the only reason htonl should get passed a negative number is that it (and possibly struct?) produce a negative number. Changing them to always produce positive numbers may be an alternative solution. Or we may want to do both, always producing positive while also accepting negative. -- Comment By: Mark Roberts (mark-roberts) Date: 2006-12-26 03:24 Message: Logged In: YES user_id=1591633 Originator: NO >From man page for htonl and friends: #include uint32_t htonl(uint32_t hostlong); uint16_t htons(uint16_t hostshort); uint32_t ntohl(uint32_t netlong); uint16_t ntohs(uint16_t netshort); Python does call these underlying functions in Modules/socketmodule.c. The problem comes from that PyLong_AsUnsignedLong() called in socket_htonl() specifically checks to see that the value cannot be less than 0. The error checking was rather exquisite, I might add. - Mark -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1619659&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
