[ python-Bugs-1586414 ] tarfile.extract() may cause file fragmentation on Windows XP
Bugs item #1586414, was opened at 2006-10-28 23:22 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1586414&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Enoch Julias (enochjul) >Assigned to: Lars Gustäbel (gustaebel) Summary: tarfile.extract() may cause file fragmentation on Windows XP Initial Comment: When I use tarfile.extract() to extract all the files from a large tar archive with lots of files tends to cause file fragmentation in Windows. Apparently NTFS cluster allocation interacts badly with such operations if Windows is not aware of the size of each file. The solution is to use a combination of the Win32 APIs SetFilePointer() and SetEndOfFile() before writing to the target file. This helps Windows choose a contiguous free space for the file. I tried it on the 2.6 trunk by calling file.truncate() (which seems to implement the appropriate calls on Windows) to set the file size before writing to a file. It helps to avoid fragmentation for the extracted files on my Windows XP x64 system. Can this be added to tarfile to improve its performance on Windows? -- Comment By: Enoch Julias (enochjul) Date: 2006-10-31 06:07 Message: Logged In: YES user_id=6071 I submitted patch #1587674 for this, though I am not sure if it is a good idea to use truncate() for such a purpose. -- Comment By: Georg Brandl (gbrandl) Date: 2006-10-29 09:55 Message: Logged In: YES user_id=849994 Can you try to come up with a patch? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1586414&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1496501 ] tarfile.py: dict order dependency
Bugs item #1496501, was opened at 2006-05-28 20:42
Message generated for change (Settings changed) made by gustaebel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1496501&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
>Assigned to: Lars Gustäbel (gustaebel)
Summary: tarfile.py: dict order dependency
Initial Comment:
When guessing the type of a file, tarfile.py tries to
open it in .tar, .tar.gz and .tar.bz2 modes
successfully. The problem is that in conjunction with a
fileobj argument, failed attempts can advance the
current position of the fileobj, and cause the following
attempts to fail as well.
This didn't show up so far because, by chance, the order
in which the OPEN_METH dictionary is enumerated is
somehow the "correct" one. The problem can be seen by
changing this order and re-running test_tarfile.py; for
example, reverse the order (tarfile.py line 1013):
for comptype in reversed(cls.OPEN_METH):
--
Comment By: Jack Diederich (jackdied)
Date: 2006-06-10 19:46
Message:
Logged In: YES
user_id=591932
added patch 1504073 because SF won't let just anyone attach
files to bugs.
--
Comment By: Jack Diederich (jackdied)
Date: 2006-06-10 18:15
Message:
Logged In: YES
user_id=591932
The invidual openers could tell() the fileobj and revert
with a seek() on failure. The module requires those methods
be implemented already to work so that is safe. I'll cook
up a patch.
But basically open() isn't as extensible as it seems.
Stream reading and writing isn't supported for injected
compression types and appending only works on plain tarfiles.
I'm guessing that isn't a problem in practice or someone
would have mentioned it by now ;)
--
Comment By: Armin Rigo (arigo)
Date: 2006-06-10 12:46
Message:
Logged In: YES
user_id=4771
If possible, I would prefer that the constructors were fixed
to become order-independent. Not sure how easy this is,
though.
--
Comment By: Jack Diederich (jackdied)
Date: 2006-06-09 21:56
Message:
Logged In: YES
user_id=591932
I took a look at tarfile.py and though there is this comment
about OPEN_METH
# This concept allows one to subclass TarFile without losing
the comfort of
# the super-constructor. A sub-constructor is registered and
made available
# by adding it to the mapping in OPEN_METH.
because adding items to OPEN_METH could change the order and
break the Tarfile.open() classmethod I doubt adding to
OPEN_METH is done in practice.
+1 for renaming OPEN_METH to _OPEN_METH and making it a list
of two-tuples.
+0 on changing the two tups from ('typename', 'funcname') to
('typename', function_ref) and dropping the getattr
functionality. The three default openers are not listed in
the public interface so if anyone is subclassing Tarfile and
overriding them (doubtful) they are doing so at their own peril.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1496501&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1527974 ] tarfile chokes on ipython archive on Windows
Bugs item #1527974, was opened at 2006-07-24 23:00 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1527974&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) >Assigned to: Lars Gustäbel (gustaebel) Summary: tarfile chokes on ipython archive on Windows Initial Comment: I'm trying to extract files from the latest ipython tar archive, available from http://ipython.scipy.org/dist/ipython-0.7.2.tar.gz, using tarfile. This is on Windows XP, using Python 2.4.3. There is only a problem if I open the archive in stream mode (the "mode" argument to tarfile.open is "r|gz"), in which case tarfile raises StreamError. I'd be happy if this error could be sorted out. The following script should trigger the error: import tarfile f = file(r"ipython-0.7.2.tar.gz", "rb") tar = tarfile.open(fileobj=f, mode="r|gz") try: for m in tar: tar.extract(m) finally: tar.close() f.close( The resulting exception: Traceback (most recent call last): File "tst.py", line 7, in ? tar.extract(m) File "C:\Program Files\Python24\lib\tarfile.py", line 1335, in extract self._extract_member(tarinfo, os.path.join(path, tarinfo.name)) File "C:\Program Files\Python24\lib\tarfile.py", line 1431, in _extract_member self.makelink(tarinfo, targetpath) File "C:\Program Files\Python24\lib\tarfile.py", line 1515, in makelink self._extract_member(self.getmember(linkpath), targetpath) File "C:\Program Files\Python24\lib\tarfile.py", line 1423, in _extract_member self.makefile(tarinfo, targetpath) File "C:\Program Files\Python24\lib\tarfile.py", line 1461, in makefile copyfileobj(source, target) File "C:\Program Files\Python24\lib\tarfile.py", line 158, in copyfileobj shutil.copyfileobj(src, dst) File "C:\Program Files\Python24\lib\shutil.py", line 22, in copyfileobj buf = fsrc.read(length) File "C:\Program Files\Python24\lib\tarfile.py", line 551, in _readnormal self.fileobj.seek(self.offset + self.pos) File "C:\Program Files\Python24\lib\tarfile.py", line 420, in seek raise StreamError, "seeking backwards is not allowed" tarfile.StreamError: seeking backwards is not allowed -- Comment By: Arve Knudsen (arve_knudsen) Date: 2006-07-27 00:20 Message: Logged In: YES user_id=1522083 Regarding my last comment, sorry about the noise. After giving it some more thought I realized it was not very realistic implementation wise, seeing as you can't know whether a file is being linked to when you encounter it in the stream (right?). So I followed your suggestion instead and handled the links on the client level. What I think I'd like to see in TarFile though is an 'extractall' method with the ability to report progress to an optional callback, since I'm only opening in stream mode as a hack to implement this myself (by monitoring file position). From browsing tarfile's source it seems it might require some effort though (with e.g. BZ2File you can't know the amount of data without decompressing everything?). -- Comment By: Arve Knudsen (arve_knudsen) Date: 2006-07-25 11:58 Message: Logged In: YES user_id=1522083 Yes I admit that is a weakness to my proposed approach. Perhaps it would be a better idea to extract hardlinked files to a temporary location and copy those files when needed, as a cache? The only problem that I can think of with this approach is the overhead, but perhaps this could be configurable through a keyword if you think it would pose a significant problem (i.e. keeping extra copies of potentially huge files)? The temporary cache would be private to tarfile, so there should be no need to worry about modifications to the contained files. -- Comment By: Lars Gustäbel (gustaebel) Date: 2006-07-25 11:31 Message: Logged In: YES user_id=642936 Copying the previously extracted file is no option. When the archive is extracted inside a loop, you never know what happens between two extract() calls. The original file could have been renamed, changed or removed. Suppose you want to extract just those members which are hard links: for tarinfo in tar: if tarinfo.islnk(): tar.extract(tarinfo) I agree with you that the error message is bad because it does not give the slightest idea of what's going wrong. I'll see what I can do about that. To work around your particular problem, my idea is to subclass the TarFile class and replace the makelink() method with one that simply copi
[ python-Bugs-1257255 ] tarfile local name is local, should be abspath
Bugs item #1257255, was opened at 2005-08-12 04:26 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1257255&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Blais (blais) >Assigned to: Lars Gustäbel (gustaebel) Summary: tarfile local name is local, should be abspath Initial Comment: I ran into a bug using the tarfile module with compression from a directory that did not exist anymore, and this exhibits a bug in the tarfile module. I'm not completely sure how to fix it with certainty, so instead of filing a quick patch, I submit a bug so people familiar with the module can have a look first. The problem is that when you open a tarfile with gz or bz2 compression, the TarFile object is opened with a filename that is not absolute, just local (basename is called) but the actual file is using a file-like object. Now, when you add a new entry, there is a check in the TarFile.add method that checks if we're not adding the archive itself into the tarfile, and this method calls abspath. Calling abspath on the name that has been basename'd is incorrect and a bug. It happens not to fail nor cause any problems when called from an existing directory (the test returns false), but it does not do the job it is supposed to. When the CWD has been deleted, calling abspath fails with an OSError, which brings out the problem. Some code to reproduce the bug is provided in an attachment. -- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-25 11:04 Message: Logged In: YES user_id=1188172 The patch has been reversed, so reopening. -- Comment By: Martin v. Löwis (loewis) Date: 2005-08-24 08:08 Message: Logged In: YES user_id=21627 This was fixed with said patch. -- Comment By: Lars Gustäbel (gustaebel) Date: 2005-08-17 14:34 Message: Logged In: YES user_id=642936 Thank you very much for your report. I added patch #1262036 that ought to fix the problem. -- Comment By: Martin Blais (blais) Date: 2005-08-15 19:34 Message: Logged In: YES user_id=10996 Oops, that was silly. My apologies. Here goes the file... -- Comment By: Lars Gustäbel (gustaebel) Date: 2005-08-15 13:39 Message: Logged In: YES user_id=642936 Could you please attach the test code you mentioned? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1257255&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1362587 ] Problem with tapedevices and the tarfile module
Bugs item #1362587, was opened at 2005-11-21 08:44
Message generated for change (Settings changed) made by gustaebel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1362587&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Henrik (esshenrik)
>Assigned to: Lars Gustäbel (gustaebel)
Summary: Problem with tapedevices and the tarfile module
Initial Comment:
Problem to read a archive from tape that was created
with the tarfile module in python.
Error:
>>> tar = tarfile.open("/dev/st0", "r")
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/local/lib/python2.4/tarfile.py", line
916, in open
return func(name, "r", fileobj)
File "/usr/local/lib/python2.4/tarfile.py", line
933, in taropen
return cls(name, mode, fileobj)
File "/usr/local/lib/python2.4/tarfile.py", line
811, in __init__
fileobj = file(self.name, self.mode)
IOError: [Errno 16] Device or resource
busy: '/dev/st0'
I was creating the archive with succes:
>>> tar.close()
>>> tar = tarfile.open("/dev/st0", "w")
>>> tar.add("/tmp/bin1.tar")
>>> tar.add("/tmp/bin2.tar")
>>> tar.close()
>>>
When I try to read the tape without pyton the
folowing error:
[EMAIL PROTECTED] tmp]# tar tvbf 20 /dev/st0
tar: Record size = 8 blocks
-rw-r--r-- root/root 128614400 2005-11-20 19:53:11
tmp/bin1.tar
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
..
Can anyone help me with this problem or have another
module for tarfile that works with tapedevices?
/Henrik
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-07-03 14:31
Message:
Logged In: YES
user_id=21627
Lowering priority, as the issue doesn't appear that urgent.
--
Comment By: Lars Gustäbel (gustaebel)
Date: 2005-11-21 09:40
Message:
Logged In: YES
user_id=642936
Could try and use tarfile's stream interface for writing and
reading and see if that works? Just append a pipe symbol to
the mode, e.g.:
tar = tarfile.open("/dev/st0", "w|")
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1362587&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-411881 ] Use of "except:" in logging module
Bugs item #411881, was opened at 2001-03-28 07:58 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=411881&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 2 Private: No Submitted By: Itamar Shtull-Trauring (itamar) >Assigned to: Vinay Sajip (vsajip) >Summary: Use of "except:" in logging module Initial Comment: A large amount of modules in the standard library use "except:" instead of specifying the exceptions to be caught. In some cases this may be correct, but I think in most cases this not true and this may cause problems. Here's the list of modules, which I got by doing: grep "except:" *.py | cut -f 1 -d " " | sort | uniq Bastion.py CGIHTTPServer.py Cookie.py SocketServer.py anydbm.py asyncore.py bdb.py cgi.py chunk.py cmd.py code.py compileall.py doctest.py fileinput.py formatter.py getpass.py htmllib.py imaplib.py inspect.py locale.py locale.py mailcap.py mhlib.py mimetools.py mimify.py os.py pdb.py popen2.py posixfile.py pre.py pstats.py pty.py pyclbr.py pydoc.py repr.py rexec.py rfc822.py shelve.py shutil.py tempfile.py threading.py traceback.py types.py unittest.py urllib.py zipfile.py -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 09:09 Message: Logged In: YES user_id=11375 Originator: NO Raymond said (in 2003) most of the remaining except: statements looked reasonable, so I'm changing this bug's summary to refer to the logging module and reassigning to vsajip. PEP 8 doesn't say anything about bare excepts; I'll bring this up on python-dev. -- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-13 06:21 Message: Logged In: YES user_id=80475 Hold-off on logging for a bit. Vinay Sajip has other patches already under review. I'll ask him to fix-up the bare excepts in conjuction with those patches. For the other modules, patches are welcome. -- Comment By: Guido van Rossum (gvanrossum) Date: 2003-12-11 15:54 Message: Logged In: YES user_id=6380 You're right. The logging module uses more blank except: clauses than I'm comfortable with. Anyone want to upload a patch set? -- Comment By: Grant Monroe (gmonroe) Date: 2003-12-11 15:50 Message: Logged In: YES user_id=929204 A good example of an incorrect use of a blanket "except:" clause is in __init__.py in the logging module. The emit method of the StreamHandler class should special case KeyboardInterrupt. Something like this: try: except KeyboardInterrupt: raise except: self.handleError(record) -- Comment By: Raymond Hettinger (rhettinger) Date: 2003-09-01 22:47 Message: Logged In: YES user_id=80475 Some efforts were made to remove many bare excepts prior to Py2.3a1. Briefly scanning those that remain, it looks like many of them are appropriate or best left alone. I recommend that this bug be closed unless someone sees something specific that demands a change. -- Comment By: Brett Cannon (bcannon) Date: 2003-05-16 19:30 Message: Logged In: YES user_id=357491 threading.py is clear. It's blanket exceptions are for printing debug output since exceptions in threads don't get passed back to the original frame anyway. -- Comment By: Skip Montanaro (montanaro) Date: 2002-08-13 23:15 Message: Logged In: YES user_id=44345 checked in fileinput.py (v 1.15) with three except:'s tightened up. The comment in the code about IOError notwithstanding, I don't see how any of the three situations would have caught anything other than OSError. -- Comment By: Skip Montanaro (montanaro) Date: 2002-08-12 15:58 Message: Logged In: YES user_id=44345 Note that this particular item was expected to be an ongoing item, with no obvious closure. Some of the bare excepts will have subtle ramifications, and it's not always obvious what specific exceptions should be caught. I've made a few changes to my local source tree which I should check in. Rather than opening new tracker items, I believe those with checkin privileges should correct those flaws they identify and attach a comment which will alert those monitoring the item. Those people without checkin privileges should just attach a patch with a note. Skip ---
[ python-Bugs-1362587 ] Problem with tapedevices and the tarfile module
Bugs item #1362587, was opened at 2005-11-21 08:44
Message generated for change (Settings changed) made by esshenrik
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1362587&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Henrik (esshenrik)
Assigned to: Lars Gustäbel (gustaebel)
Summary: Problem with tapedevices and the tarfile module
Initial Comment:
Problem to read a archive from tape that was created
with the tarfile module in python.
Error:
>>> tar = tarfile.open("/dev/st0", "r")
Traceback (most recent call last):
File "", line 1, in ?
File "/usr/local/lib/python2.4/tarfile.py", line
916, in open
return func(name, "r", fileobj)
File "/usr/local/lib/python2.4/tarfile.py", line
933, in taropen
return cls(name, mode, fileobj)
File "/usr/local/lib/python2.4/tarfile.py", line
811, in __init__
fileobj = file(self.name, self.mode)
IOError: [Errno 16] Device or resource
busy: '/dev/st0'
I was creating the archive with succes:
>>> tar.close()
>>> tar = tarfile.open("/dev/st0", "w")
>>> tar.add("/tmp/bin1.tar")
>>> tar.add("/tmp/bin2.tar")
>>> tar.close()
>>>
When I try to read the tape without pyton the
folowing error:
[EMAIL PROTECTED] tmp]# tar tvbf 20 /dev/st0
tar: Record size = 8 blocks
-rw-r--r-- root/root 128614400 2005-11-20 19:53:11
tmp/bin1.tar
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: /dev/st0: Cannot read: Cannot allocate memory
..
Can anyone help me with this problem or have another
module for tarfile that works with tapedevices?
/Henrik
--
Comment By: Martin v. Löwis (loewis)
Date: 2006-07-03 14:31
Message:
Logged In: YES
user_id=21627
Lowering priority, as the issue doesn't appear that urgent.
--
Comment By: Lars Gustäbel (gustaebel)
Date: 2005-11-21 09:40
Message:
Logged In: YES
user_id=642936
Could try and use tarfile's stream interface for writing and
reading and see if that works? Just append a pipe symbol to
the mode, e.g.:
tar = tarfile.open("/dev/st0", "w|")
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1362587&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-436259 ] [Windows] exec*/spawn* problem with spaces in args
Bugs item #436259, was opened at 2001-06-25 23:17
Message generated for change (Comment added) made by akuchling
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=436259&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ben Hutchings (wom-work)
Assigned to: Nobody/Anonymous (nobody)
Summary: [Windows] exec*/spawn* problem with spaces in args
Initial Comment:
DOS and Windows processes are not given an argument
vector, as Unix processes are; instead they are given a
command line and are expected to perform any necessary
argument parsing themselves. Each C run-time library
must convert command lines into argument vectors for
the main() function, and if it includes exec* and
spawn* functions then those must convert argument
vectors into a command-line. Naturally, the various
implementations differ in interesting ways.
The Visual C++ run-time library (MSVCRT) implementation
of the exec* and spawn* functions is particularly awful
in that it simply concatenates the strings with spaces
in-between (see source file cenvarg.c), which means
that arguments with embedded spaces are likely to turn
into multiple arguments in the new process. Obviously,
when Python is built using Visual C++, its os.exec* and
os.spawn* functions behave in this way too. MS prefers
to work around this bug (see Knowledge Base article
Q145937) rather than to fix it. Therefore I think
Python must work around it too when built with Visual C++.
I experimented with MSVCRT and Cygwin (using the
attached program print_args.c) and could not find a way
to convert an argument vector into a command line that
they would both convert back to the same argument
vector, but I got close.
MSVCRT's parser requires spaces that are part of an
argument to be enclosed in double-quotes. The
double-quotes do not have to enclose the whole
argument. Literal double-quotes must be escaped by
preceding them with a backslash. If an argument
contains literal backslashes before a literal or
delimiting double-quote, those backslashes must be
escaped by doubling them. If there is an unmatched
enclosing double-quote then the parser behaves as if
there was another double-quote at the end of the line.
Cygwin's parser requires spaces that are part of an
argument to be enclosed in double-quotes. The
double-quotes do not have to enclose the whole
argument. Literal double-quotes may be escaped by
preceding them with a backslash, but then they count as
enclosing double-quote as well, which appears to be a
bug. They may also be escaped by doubling them, in
which case they must be enclosed in double-quotes;
since MSVCRT does not accept this, it's useless. As far
as I can see, literal backslashes before a literal
double-quote must not be escaped and literal
backslashes before an enclosing double-quote *cannot*
be escaped. It's really quite hard to understand what
its rules are for backslashes and double-quotes, and I
think it's broken. If there is an unmatched enclosing
double-quote then the parser behaves as if there was
another double-quote at the end of the line.
Here's a Python version of a partial fix for use in
nt.exec* and nt.spawn*. This function modifies
argument strings so that the resulting command line
will satisfy programs that use MSVCRT, and programs
that use Cygwin if that's possible.
def escape(arg):
import re
# If arg contains no space or double-quote then
# no escaping is needed.
if not re.search(r'[ "]', arg):
return arg
# Otherwise the argument must be quoted and all
# double-quotes, preceding backslashes, and
# trailing backslashes, must be escaped.
def repl(match):
if match.group(2):
return match.group(1) * 2 + '\\"'
else:
return match.group(1) * 2
return '"' + re.sub(r'(\\*)("|$)', repl, arg) + '"'
This could perhaps be used as a workaround for the
problem. Unfortunately it would conflict with
workarounds implemented at the Python level (which I
have been using for a while).
--
>Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-21 09:48
Message:
Logged In: YES
user_id=11375
Originator: NO
Does this argument-line parsing weirdness still have relevance to current
MSVC runtimes?
Changing os.spawn() seems like a non-starter because it'll break existing
code; the Python landscape has changed and subprocess.py is a
higher-level, more useful way to run subprocesses (it has a MS C
runtime-alike function, list2cmdline).
Unless someone submits a patch to change _nt_quote_args in
distutils/spawn.py, I'll close this bug in a few months (the next time I
visit the really old bugs).
--
[ python-Bugs-469773 ] Write 'Using Python on Platform X' documents
Bugs item #469773, was opened at 2001-10-10 02:28 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Private: No Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Write 'Using Python on Platform X' documents Initial Comment: Some stuff that's in the man page (e.g. the syntax for the -W switch) doesn't appear to be in the HTML documentation. If it is there, it's _very_ well hidden away. -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 09:50 Message: Logged In: YES user_id=11375 Originator: NO Changing summary to be more accurate. -- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-11-11 02:27 Message: Logged In: YES user_id=29957 What about instead a new "Using Python" document. There's a bunch of overlap between the platforms. We can then also have separate chapters for Mac, Win, Unix. I think trying to produce platform specific docs is going to be a headache in the long run. -- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-11-11 01:33 Message: Logged In: YES user_id=3066 I'll note that the "Macintosh Library Modules" document deals with this sort of issue for the Mac (though it may be out of date for Mac OS X; not sure). The first chapter is an introduction to using Python on the Mac, and the remaining chapters provide module reference documentation. What's needed is something like the first chapter of that for Unix and Windows. I'm not sure where it should go, though; I don't want that material in the main library reference. This almost makes me want three platform-specific documents, one for Windows, one for Unix, and the existing Mac OS document, each containing the relevant "Using Python on ..." chapter, followed by chapters containing platform-specific modules. Modules that apply to two or more of the platforms would continue to be documented in the main library reference. The distutils modules are already documented in the distutils documentation as well. This should be fine since the "global" module index is the normal general entry point anyway. -- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-25 02:17 Message: Logged In: YES user_id=29957 See, that's the problem :) It's not clear where this might go - it's almost a "Using Python" section, which, as far as I can see, doesn't really exist. This might also include stuff about the Windows GUI and other interfaces for using python. If a shorter-term "just get the -W stuff in" approach is wanted, then extended the lib/module-warnings might be the approach... but it really does seem like it needs to be somewhere central. -- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 22:12 Message: Logged In: YES user_id=3066 I agree. Where would you suggest the information be added? -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=469773&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-502236 ] Asynchronous exceptions between threads
Feature Requests item #502236, was opened at 2002-01-11 03:46 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=502236&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Extension Modules >Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) >Summary: Asynchronous exceptions between threads Initial Comment: I may be missing something but there doesn't seem to be any easy, correct way to exit a Python application and shut down the interpreter. sys.exit() raises the SysExit exception in the current thread but other threads keep running, so the application doesn't stop. You can do something brutal like os.kill() your process, but then cleanup actions (finally clauses) don't get run. Or you can create some elaborate application-specific framework for propagating an exit flag from one thread to all the rest. That's incredibly lame. What's needed is a simple function (maybe sys.allexit) that raises SysExit in all the threads of the application (it might have to first stop all the threads using the GIL or something like that). I'm surprised there isn't already something like this, but there's been some c.l.py discussion about it and it really seems to not be there. Is there some problem with the idea? I don't see one but I can't be sure. -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 09:52 Message: Logged In: YES user_id=11375 Originator: NO Changing summary to be clearer; reclassifying as 'Feature Request' type. -- Comment By: paul rubin (phr) Date: 2002-01-15 14:40 Message: Logged In: YES user_id=72053 I don't feel qualified to write such a PEP. You guys are more on top of this than I am. Maybe there's no portable way to do asynchronous exceptions between threads. I don't know what happens if you hit ctrl-C when running a multi- threaded Linux program. I don't think ctrl-C even works in Windows--all you can really do is bring up the task manager and blow away the whole process. But I think asynchronous exceptions are worth having even if they're OS specific and only work on some systems. As for how a thread can exit the whole program, I thought Guido's first message (saying raise some asynchronous exception in all the threads) sounded fine, if it's feasible. -- Comment By: Guido van Rossum (gvanrossum) Date: 2002-01-15 14:33 Message: Logged In: YES user_id=6380 (1) I think subclassing SystemExit can work. (2) But the difference is that killing the main thread without giving it a chance to clean up is worse than killing another thread. Does anybody want to write a PEP on thread cancellation? It sure looks like a hairy issue! -- Comment By: Martin v. Löwis (loewis) Date: 2002-01-14 03:40 Message: Logged In: YES user_id=21627 On (1): This is quite unfortunate, as I do think sys.exit and thread.exit should do different things. There would be approaches of further subclassing SystemExit; preserving the property that raising SystemExit in a thread exits the thread only - would such a solution be acceptable? On (2): This is no different from raising SystemExit in the main thread, which also does not invoke finally clauses in other threads. I don't think there can be a complete, reliable implementation of thread cancellation, only a best-effort approach. For POSIX, I'd suggest sending SIGUSR2 to each thread through pthread_kill. We might need to add a mechanism indicating whether a thread is ready to be cancelled, similar to the POSIX cancellation points. That would prevent the signal from arriving in an arbitrary library function, and defer thread termination until the library function completes. For blocking system calls, sending signals would need to be activated explicitly, to allow aborting them. -- Comment By: Guido van Rossum (gvanrossum) Date: 2002-01-12 15:03 Message: Logged In: YES user_id=6380 (1) It's too late to change the meaning of SystemExit -- people are relying on SystemExit exiting the thread only. (2) If we provide a way for a thread to exit the whole program, how will finally clauses in other threads (including the main thread) be executed? -- Comment By: Martin v. Löwis (loewis) Date: 2002-01-11 20:05 Message: Logged In: YES user_id=21627 Tim, I'm not surprised that raising SystemExit in a threa
[ python-Bugs-539175 ] resolver not thread safe
Bugs item #539175, was opened at 2002-04-04 04:54 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539175&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: dustin sallings (dustin) Assigned to: Nobody/Anonymous (nobody) Summary: resolver not thread safe Initial Comment: I've got an application that does SNMP monitoring and has a thread listening with SimpleXMLRPCServer for remote control. I noticed the XMLRPC listener logging an incorrect address while snmp jobs were processing: sw1.west.spy.net - - [04/Apr/2002 01:16:37] "POST /RPC2 HTTP/1.0" 200 - localhost.west.spy.net - - [04/Apr/2002 01:16:43] "POST /RPC2 HTTP/1.0" 200 - sw1 is one of the machines that is being queried, but the XMLRPC requests are happening over localhost. gethostbyname() and gethostbyaddr() both return static data, thus they aren't reentrant. As a workaround, I copied socket.py to my working directory and added the following to it: try: import threading except ImportError, ie: sys.stderr.write(str(ie) + "\n") # mutex for DNS lookups __dns_mutex=None try: __dns_mutex=threading.Lock() except NameError: pass def __lock(): if __dns_mutex!=None: __dns_mutex.acquire() def __unlock(): if __dns_mutex!=None: __dns_mutex.release() def gethostbyaddr(addr): """Override gethostbyaddr to try to get some thread safety.""" rv=None try: __lock() rv=_socket.gethostbyaddr(addr) finally: __unlock() return rv def gethostbyname(name): """Override gethostbyname to try to get some thread safety.""" rv=None try: __lock() rv=_socket.gethostbyname(name) finally: __unlock() return rv -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 10:13 Message: Logged In: YES user_id=11375 Originator: NO Attaching the test script. The script now fails because some of the spy.net addresses are resolved to hostnames such as adsl-69-230-8-158.dsl.pltn13.pacbell.net. When I changed the test script to use python.org machine names and ran it with Python 2.5 on Linux, no errors were reported. Does this still fail on current OS X? If not, I suggest calling this a platform C library bug and closing this report. File Added: resolv-bug.py -- Comment By: dustin sallings (dustin) Date: 2002-08-11 15:27 Message: Logged In: YES user_id=43919 No, unfortunately, I haven't been able to look at it in a while. Short of locking it in python, I wasn't able to avoid the failure. I'm sorry I haven't updated this at all. As far as I can tell, it's still a problem, but I haven't not been able to find a solution in the C code. I supposely I spoke with too much haste when I said I was perfectly capable of fixing the problem myself. The locking in the C code did seem correct, but the memory was still getting stomped. -- Comment By: Neal Norwitz (nnorwitz) Date: 2002-08-11 11:04 Message: Logged In: YES user_id=33168 Dustin, any progress on a patch or diagnosing this further? -- Comment By: dustin sallings (dustin) Date: 2002-04-05 16:44 Message: Logged In: YES user_id=43919 I first noticed this problem on my OS X box. Since it's affecting me, it's not obvious to anyone else, and I'm perfectly capable of fixing it myself, I'll try to spend some time figuring out what's going on this weekend. It seems like it might be making a decision to not use the lock at compile time. I will investigate further and submit a patch. -- Comment By: Tim Peters (tim_one) Date: 2002-04-05 16:31 Message: Logged In: YES user_id=31435 Just a reminder that the first thing to try on any SGI box is to recompile Python with optimization disabled. I can't remember the last time we had "a Python bug" on SGI that wasn't traced to a compiler -O bug. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 03:56 Message: Logged In: YES user_id=21627 Can you spot the error in the Python socket module? I still fail to see our bug, and I would assume it is a C library bug; I also cannot reproduce the problem on any of my machines. Can you please report the settings of the various HAVE_ defines for irix? -- Comment By: dustin sallings (dustin)
[ python-Feature Requests-665194 ] datetime-RFC2822 roundtripping
Feature Requests item #665194, was opened at 2003-01-09 13:24 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=665194&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None >Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter Dörwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: datetime-RFC2822 roundtripping Initial Comment: It would be good to have a simply way to convert between datetime objects and RFC2822 style strings. From string to datetime is easy with datetime.datetime(*email.Utils.parsedate(m)[:7]) (but this drops the timezone), but the other direction seems impossible. email.Utils.formatdate takes a timestamp argument, but AFAICT there's no way to get a timestamp out of a datetime object. Of course the best solution (ignoring backwards compatibility) would be for parsedate und formatdate to return/accept datetime objects or for datetime to have the appropriate methods. -- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 10:16 Message: Logged In: YES user_id=11375 Originator: NO Moving to feature requests. -- Comment By: Walter Dörwald (doerwalter) Date: 2003-05-26 12:56 Message: Logged In: YES user_id=89016 time.strptime() is locale aware, but RFC1123 & RFC822 require english day and month names, so time.strptime() can't be used as-is. (And of course time.strptime() only works for formatting, not for parsing) -- Comment By: Brett Cannon (bcannon) Date: 2003-05-23 18:46 Message: Logged In: YES user_id=357491 time.strptime doesn't solve your problem? -- Comment By: Walter Dörwald (doerwalter) Date: 2003-01-13 08:05 Message: Logged In: YES user_id=89016 totimestamp() should be the inverse of fromtimestamp(), i.e. foo.totimestamp() should be the same as time.mktime(foo.timetuple()). datetime.datetime.totimestamp() should raise OverflowError iff time.mktime() raises OverflowError. But as this may lose precision, I'd say it would be better, if datetime supported RFC1123 conversion directly, i.e. two methods frommime() and tomime(), that parse and format strings like "Sun, 06 Nov 1994 08:49:37 GMT" (what rfc822.parsedate() and rfc822.formatdate() do) -- Comment By: Tim Peters (tim_one) Date: 2003-01-11 11:32 Message: Logged In: YES user_id=31435 Define what totimestamp() should do. The range of timestamps supported by the *platform* C library (and so indirectly by Python's time module) isn't defined by any standard, and isn't easily discoverable either. It may or may not work before 1970, may or may not after 2038. datetime covers days far outside that range. Note that even a double doesn't have enough bits of precision to cover the full range of datetime values, either. In contrast, ordinals are wholly under Python's control, so we can promise surprise-free conversion in both directions. All we can promise about timestamps is that if the platform supports a timestamp for a time in datetime's range, datetime can make sense of it. -- Comment By: Walter Dörwald (doerwalter) Date: 2003-01-11 07:33 Message: Logged In: YES user_id=89016 OK, I'll mark this a feature request. datetime has fromordinal() and toordinal(), it has fromtimestamp(), so I'd say a totimestamp() method would be a logical addition. -- Comment By: Tim Peters (tim_one) Date: 2003-01-11 00:10 Message: Logged In: YES user_id=31435 You can get a timestamp like so: >>> time.mktime(datetime.date(2002, 1, 1).timetuple()) 1009861200.0 >>> The dates for which this works depends on the platform mktime implementation, though. BTW, this sounds a lot more like a new feature request than a bug! -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=665194&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-665761 ] reduce() masks exception
Bugs item #665761, was opened at 2003-01-10 09:41
Message generated for change (Comment added) made by akuchling
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665761&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 3
Private: No
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Nobody/Anonymous (nobody)
Summary: reduce() masks exception
Initial Comment:
In the following test script
-
class Test:
def __iter__(self):
raise IOError
reduce(lambda x,y: x+y, Test())
-
the real IOError exception is masked, i.e. the traceback is
-
Traceback (most recent call last):
File "test.py", line 5, in ?
reduce(lambda x,y: x+y, Test())
TypeError: reduce() arg 2 must support iteration
-
but IMHO should be
-
Traceback (most recent call last):
File "test.py", line 3, in ?
raise IOError
IOError
-
This can be fixed by removing the
PyErr_SetString(PyExc_TypeError, "reduce() arg 2 must
support iteration") call in
bltinmodule.c/buildtin_reduce().
--
>Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-21 10:35
Message:
Logged In: YES
user_id=11375
Originator: NO
Preserving the argument number seems difficult without exception chaining;
perhaps a middle group would be to only replace TypeError exceptions.
Walter's comment at 2003-01-30 06:29 shows this can still be fooled, but a
fix for 90% of the cases is still better than a fix for 0% of them.
The patch looks OK, but I think it should be reworked to take the
middle-ground approach, replacing only TypeErrors.
--
Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-30 06:29
Message:
Logged In: YES
user_id=89016
Trapping only TypeError won't help:
class LazyFile:
def __init__(self, name, mode="r"):
self.name = name
self.mode = mode
def __iter__(self):
return open(self.name, self.mode)
import operator
f = LazyFile(42)
s = reduce(operator.add, f)
Here the open call will raise a TypeError, that is totally
unrelated to the iterator protocol.
The cleanest solution would really be exception chaining.
--
Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-28 18:59
Message:
Logged In: YES
user_id=80475
One way to mitigate the information loss is to mimic the
style in zip() which only traps TypeErrors but passes
through things like the IOError in your original example.
--
Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-28 15:38
Message:
Logged In: YES
user_id=89016
Attached is a patch that fixes reduce(), map() and zip().
Unfortunately this loses the information about which
argument triggered the exception (in map() and zip())
--
Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 23:00
Message:
Logged In: YES
user_id=80475
There's a lot of this going around. map() and zip() have
the same issue. I recommend fixing them all.
--
Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-13 08:18
Message:
Logged In: YES
user_id=89016
The point is that Python/bltinmodule.c/builtin_reduce()
masks the error returned from PyObject_GetIter(). Errors
from PyIter_Next() are not masked.
What about the following example:
class LazyFile:
def __init__(self, name, mode="r"):
self.name = name
self.mode = mode
def __iter__(self):
return open(self.name, self.mode)
import operator
f = LazyFile("does not exist")
s = reduce(operator.add, f)
LazyFile *does* support iteration, but the underlying
problem of the non existing file is masked. Removing the
call PyErr_SetString(PyExc_TypeError, "reduce() arg 2 must
support iteration"); in builtin_reduce(), will produce the
original exception "IOError: [Errno 2] No such file or
directory: 'does not exist'" and when the second argument is
not iteratable, the original exception is just as good:
>>> reduce(lambda x,y: x+y, 42)
Traceback (most recent call last):
File "", line 1, in ?
TypeError: iteration over non-sequence
--
Comment By: Jp Calderone (kuran)
Date: 2003-01-11 12:08
Message:
Logged In: YES
user_id=366566
the __iter__ method is supposed to return an object that
defines a 'next' method. The returned object is the one
used for iteration, not the original. So I believe the
error message is correct - Test does not support iteration.
If you chan
[ python-Bugs-539175 ] resolver not thread safe
Bugs item #539175, was opened at 2002-04-04 01:54 Message generated for change (Comment added) made by dustin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539175&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: None >Status: Closed >Resolution: Works For Me Priority: 5 Private: No Submitted By: dustin sallings (dustin) Assigned to: Nobody/Anonymous (nobody) Summary: resolver not thread safe Initial Comment: I've got an application that does SNMP monitoring and has a thread listening with SimpleXMLRPCServer for remote control. I noticed the XMLRPC listener logging an incorrect address while snmp jobs were processing: sw1.west.spy.net - - [04/Apr/2002 01:16:37] "POST /RPC2 HTTP/1.0" 200 - localhost.west.spy.net - - [04/Apr/2002 01:16:43] "POST /RPC2 HTTP/1.0" 200 - sw1 is one of the machines that is being queried, but the XMLRPC requests are happening over localhost. gethostbyname() and gethostbyaddr() both return static data, thus they aren't reentrant. As a workaround, I copied socket.py to my working directory and added the following to it: try: import threading except ImportError, ie: sys.stderr.write(str(ie) + "\n") # mutex for DNS lookups __dns_mutex=None try: __dns_mutex=threading.Lock() except NameError: pass def __lock(): if __dns_mutex!=None: __dns_mutex.acquire() def __unlock(): if __dns_mutex!=None: __dns_mutex.release() def gethostbyaddr(addr): """Override gethostbyaddr to try to get some thread safety.""" rv=None try: __lock() rv=_socket.gethostbyaddr(addr) finally: __unlock() return rv def gethostbyname(name): """Override gethostbyname to try to get some thread safety.""" rv=None try: __lock() rv=_socket.gethostbyname(name) finally: __unlock() return rv -- >Comment By: dustin sallings (dustin) Date: 2006-12-21 20:24 Message: Logged In: YES user_id=43919 Originator: YES I'll go ahead and close it. It does not fail under 2.4 on any of my machines (tried OS X/intel, PPC G3, and FreeBSD/intel). -- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 07:13 Message: Logged In: YES user_id=11375 Originator: NO Attaching the test script. The script now fails because some of the spy.net addresses are resolved to hostnames such as adsl-69-230-8-158.dsl.pltn13.pacbell.net. When I changed the test script to use python.org machine names and ran it with Python 2.5 on Linux, no errors were reported. Does this still fail on current OS X? If not, I suggest calling this a platform C library bug and closing this report. File Added: resolv-bug.py -- Comment By: dustin sallings (dustin) Date: 2002-08-11 12:27 Message: Logged In: YES user_id=43919 No, unfortunately, I haven't been able to look at it in a while. Short of locking it in python, I wasn't able to avoid the failure. I'm sorry I haven't updated this at all. As far as I can tell, it's still a problem, but I haven't not been able to find a solution in the C code. I supposely I spoke with too much haste when I said I was perfectly capable of fixing the problem myself. The locking in the C code did seem correct, but the memory was still getting stomped. -- Comment By: Neal Norwitz (nnorwitz) Date: 2002-08-11 08:04 Message: Logged In: YES user_id=33168 Dustin, any progress on a patch or diagnosing this further? -- Comment By: dustin sallings (dustin) Date: 2002-04-05 13:44 Message: Logged In: YES user_id=43919 I first noticed this problem on my OS X box. Since it's affecting me, it's not obvious to anyone else, and I'm perfectly capable of fixing it myself, I'll try to spend some time figuring out what's going on this weekend. It seems like it might be making a decision to not use the lock at compile time. I will investigate further and submit a patch. -- Comment By: Tim Peters (tim_one) Date: 2002-04-05 13:31 Message: Logged In: YES user_id=31435 Just a reminder that the first thing to try on any SGI box is to recompile Python with optimization disabled. I can't remember the last time we had "a Python bug" on SGI that wasn't traced to a compiler -O bug. -- Comment By: Martin v. Löwis (loewis) Date: 2002-04-05 00:56 Message: Logged In: YES user_id=21627 Can you spot the error in the Python socket
[ python-Feature Requests-1618676 ] Promote list comprehensions to Language Integrated Query?
Feature Requests item #1618676, was opened at 2006-12-19 02:08
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1618676&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: JettLogic (jettlogic)
Assigned to: Nobody/Anonymous (nobody)
Summary: Promote list comprehensions to Language Integrated Query?
Initial Comment:
Hi,
A feature idea (shall put to mailing list discussion sometime), is to extend
list comprehensions to many uses of SQL or XQuery unnecessary.
The benefit of language integration over passing SQL/XQuery strings to external
libraries would be making code agnostic to the source of the data: in-memory
tables and trees could be queried just like a relational DB or XML. An "order
by" could be more efficient than sorted(f(x) for x in y).
Of course list comprehensions do much of the C#-style LINQ already, but they
can't (easily) do joins or "select x from a, y from b" type queries, or return
XML a la XQuery.
On the library side, DBAPI helps make code independent of any particular
database, and sqlite in 2.5 means in-memory tables (lists of tuples) aren't
really necessary.
If worthwhile, implementation would probably be best suited to PyPy.
C# Example from http://en.wikipedia.org/wiki/Language_Integrated_Query
var q = from o in db.Orders, c in db.Customers
where o.Quality == "200" && (o.CustomerID == c.CustomerID)
select new { o.DueDate, c.CompanyName, c.ItemID, c.ItemName };
--
Comment By: Josiah Carlson (josiahcarlson)
Date: 2006-12-21 21:59
Message:
Logged In: YES
user_id=341410
Originator: NO
There have been various discussions about metasyntax proposals and Python,
see the Python-3000 mailing list for references to LINQ. The general
answer has been "no". Note that you can do much of what you want by
overloading operations on standard Python objects like __or__, __and__,
etc.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1618676&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-411881 ] Use of "except:" in logging module
Bugs item #411881, was opened at 2001-03-28 12:58 Message generated for change (Comment added) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=411881&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 2 Private: No Submitted By: Itamar Shtull-Trauring (itamar) Assigned to: Vinay Sajip (vsajip) Summary: Use of "except:" in logging module Initial Comment: A large amount of modules in the standard library use "except:" instead of specifying the exceptions to be caught. In some cases this may be correct, but I think in most cases this not true and this may cause problems. Here's the list of modules, which I got by doing: grep "except:" *.py | cut -f 1 -d " " | sort | uniq Bastion.py CGIHTTPServer.py Cookie.py SocketServer.py anydbm.py asyncore.py bdb.py cgi.py chunk.py cmd.py code.py compileall.py doctest.py fileinput.py formatter.py getpass.py htmllib.py imaplib.py inspect.py locale.py locale.py mailcap.py mhlib.py mimetools.py mimify.py os.py pdb.py popen2.py posixfile.py pre.py pstats.py pty.py pyclbr.py pydoc.py repr.py rexec.py rfc822.py shelve.py shutil.py tempfile.py threading.py traceback.py types.py unittest.py urllib.py zipfile.py -- >Comment By: Vinay Sajip (vsajip) Date: 2006-12-22 07:42 Message: Logged In: YES user_id=308438 Originator: NO The reason for the fair number of bare excepts in logging is this: in many cases (e.g. long-running processes like Zope servers) users don't want their application to change behaviour just because of some exception thrown in logging. So, logging aims to be very quiet indeed and swallows exceptions, except SystemExit and KeyboardInterrupt in certain situations. Also, logging is one of the modules which is (meant to be) 1.5.2 compatible, and string exceptions are not that uncommon in older code. I've looked at bare excepts in logging and here's my summary on them: logging/__init__.py: currentframe(): Backward compatibility only, sys._getframe is used where available so currentframe() will only be called on rare occasions. LogRecord.__init__(): There's a try/bare except around calls to os.path.basename() and os.path.splitext(). I could add a raise clause for SystemExit/KeyboardInterrupt. StreamHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). shutdown(): Normally only called at system exit, and will re-raise everything if raiseExceptions is set (the default). logging/handlers.py: BaseRotatingHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). SocketHandler.createSocket(): I could add a raise clause for SystemExit/KeyboardInterrupt. SocketHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). SysLogHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). SMTPHandler.emit(): Should change bare except to ImportError for the formatdate import. Elsewhere, reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). NTEventLogHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). HTTPHandler.emit(): Reraises SystemExit and KeyboardInterrupt, and otherwise calls handleError() which raises everything if raiseExceptions is set (the default). logging/config.py: listen.ConfigStreamHandler.handle(): Reraises SystemExit and KeyboardInterrupt, prints everything else and continues - seems OK for a long-running thread. What do you think? -- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-21 14:09 Message: Logged In: YES user_id=11375 Originator: NO Raymond said (in 2003) most of the remaining except: statements looked reasonable, so I'm changing this bug's summary to refer to the logging module and reassigning to vsajip. PEP 8 doesn't say anything about bare excepts; I'll bring this up on python-dev. -- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-13 11:21 Message: Logged In: YES user_id=80475 Hold-off on logging for a bit. Vinay Sajip has other patches already under review. I'll ask him to fix-up the bare ex
[ python-Bugs-1611154 ] os.path.exists("file/") failure on Solaris 9
Bugs item #1611154, was opened at 2006-12-07 22:25
Message generated for change (Comment added) made by eggert
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611154&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Paul Eggert (eggert)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.exists("file/") failure on Solaris 9
Initial Comment:
Solaris 9 and earlier fail to conform to POSIX, in that
stat("FILE/") succeeds even when FILE is not a directory.
POSIX says that in this case it should fail. This problem causes
os.path.exists("FILE/") to succeed when it should fail, which makes it harder
to write portable Python code. One of my students ran into this problem when
doing a Django-based project: his code ran fine on his Linux box, but failed
when he attempted to run it on the Solaris 8 server that is the standard
platform for our students.
To reproduce the problem, on Solaris 8 (or 9):
$ touch file
$ ls -l file
-rw-rw-r-- 1 eggert csfac 0 Dec 7 14:19 file
$ python
Python 2.5 (r25:51908, Dec 7 2006, 13:14:10)
[GCC 4.1.1] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists("file/")
True
It should be False.
I'll attach a patch that works around the problem at
run-time. If you prefer something that tests for it
at compile time please let me know.
--
>Comment By: Paul Eggert (eggert)
Date: 2006-12-22 07:56
Message:
Logged In: YES
user_id=17848
Originator: YES
OK, I'll attach a revised patch that uses a configure test to check for
this bug.
File Added: python-stat-patch-2.diff
--
Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 13:37
Message:
Logged In: YES
user_id=11375
Originator: NO
I think it would be sufficient to put a #ifdef ... #endif
around the additional check. We don't want to make all platforms do extra
system calls in order to avoid a Solaris 9 bug. Or you could write a
configure test to check for this bug, but that's more complicated a task.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1611154&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
