[ python-Bugs-1710703 ] zipfile.ZipFile behavior inconsistent.
Bugs item #1710703, was opened at 2007-05-01 11:43
Message generated for change (Comment added) made by markaflacy
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710703&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Flacy (markaflacy)
Assigned to: Nobody/Anonymous (nobody)
Summary: zipfile.ZipFile behavior inconsistent.
Initial Comment:
In short, ZipFile() will not write the Central Directory entry unless you have
added a file to it. That makes it impossible to create a valid empty zip
archive.
In one of my applications, I have the need to extract a partial set of
information from one zip file and insert it into another. There are valid use
cases where the source zip archive will not have any of the files which I am
looking for. In Python 2.4, I would end up with an empty file which was
considered to be a valid empty zip archive. In Python 2.5, an empty file is
not considered a valid zip archive.
One would reasonably expect that creating a new ZipFile(mode="r") and
successfully closing it without writing any entries would result in a valid zip
archive that could be re-opened later without throwing an exception.
--
>Comment By: Mark Flacy (markaflacy)
Date: 2007-05-04 02:26
Message:
Logged In: YES
user_id=1344993
Originator: YES
No wonder you're confused. My description of how 2.4 worked was flat-out
wrong; empty files opened as zip files will throw IOExceptions and have
done so since 2.4 at least (I didn't look further back than that).
However, it *is* the case that 2.4 would correctly write the Central
Directory entry on zipfile close for "w" and "a" modes, even for zip files
that never had any entries written into them.
In 2.4, the ZipFile.close() method contains the line...
if self.mode in ("w", "a"): # write ending records
...while in 2.5, the test was changed to...
if self.mode in ("w", "a") and self._didModify: # write ending
records
That change was added in revision 46967 as part of the ZIP64 support and
that change breaks backwards compatibility (as well as not making a lot of
sense for the "w" case).
--
Comment By: Neal Norwitz (nnorwitz)
Date: 2007-05-02 01:06
Message:
Logged In: YES
user_id=33168
Originator: NO
Mark, can you create a patch for zipfile to make it do what you want? Do
the docs mention anything about this either way? Perhaps the docs also
need updating?
I don't know about what happened here, but I'm guessing there was some bug
fix. This change could have been intentional or not. I patch will help us
figure out what went wrong and how to proceed.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710703&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1712419 ] Cannot use dict with unicode keys as keyword arguments
Feature Requests item #1712419, was opened at 2007-05-04 00:49
Message generated for change (Comment added) made by lemburg
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1712419&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Viktor Ferenczi (complex)
>Assigned to: Nobody/Anonymous (nobody)
Summary: Cannot use dict with unicode keys as keyword arguments
Initial Comment:
Unicode strings cannot be used as keys in dictionaries passed as keyword
argument to a function. For example:
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def fn(**kw):
... print repr(kw)
...
>>> fn(**{u'x':1})
Traceback (most recent call last):
File "", line 1, in
TypeError: fn() keywords must be strings
>>>
Unicode strings should be converted to str automatically using the site
specific default encoding or something similar solution.
This bug caused problems when decoding dictionaries from data formats such as
XML or JSON. Usually unicode strings are returned from such modules, such as
simplejson. Manual encoding from unicode to str causes performance loss if this
cannot be done by Python automatically.
--
>Comment By: M.-A. Lemburg (lemburg)
Date: 2007-05-04 12:27
Message:
Logged In: YES
user_id=38388
Originator: NO
Python (currently) does not allow non-ASCII identifiers, so it's not
surprising that Unicode parameter names don't work.
It's also a really bad idea to pass data from an AJAX XML or JSON request
directly to a function without doing at least some post-processing of the
data in order to prevent DOS attacks, code injection, etc.
dict((str(key), value) for key, value in kw.iteritems()) is all that's
needed for the above.
BTW, I don't think those few micro-seconds really matter in the face of
XML or JSON decoding, network latency, etc. ;-)
--
Comment By: Georg Brandl (gbrandl)
Date: 2007-05-04 06:10
Message:
Logged In: YES
user_id=849994
Originator: NO
In any case, this is a feature request.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1712419&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1712742 ] pprint handles depth argument incorrectly
Bugs item #1712742, was opened at 2007-05-04 16:47 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1712742&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Dmitrii Tisnek (dimaq) Assigned to: Nobody/Anonymous (nobody) Summary: pprint handles depth argument incorrectly Initial Comment: pprint.pprint( [ ], depth=0) AssertionError: depth must be > 0 pprint.pprint( [ ], depth=0.5) [[...]] pprint.pprint( [ ], depth=1) [[[...]]] I would like to see the root (that is [...]) with depth=0 (or depth=1 if you insist), either way, [[[...]]] for depth=1 is incorrect. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1712742&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1581906 ] test_sqlite fails on OSX G5 arch if test_ctypes is run
Bugs item #1581906, was opened at 2006-10-21 20:02 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1581906&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Skip Montanaro (montanaro) Assigned to: Skip Montanaro (montanaro) Summary: test_sqlite fails on OSX G5 arch if test_ctypes is run Initial Comment: I noticed a test_sqlite test failure on my Mac G5 the other day while trying to set up a buildbot for sqlalchemy. Everything runs fine on my G4 powerbook. Both machines run Mac OSX 10.4.8 with Apple's gcc 4.0.0, build 5026. I whittled the problem down to just having test_ctypes and test_sqlite enabled, then further whittled it down to just having Lib/ctypes/test/ test_find.py available (all other ctypes tests eliminated). More detailed problem descriptions are in these two postings to python-dev: http://article.gmane.org/gmane.comp.python.devel/84478 http://article.gmane.org/gmane.comp.python.devel/84481 Skip -- >Comment By: Thomas Heller (theller) Date: 2007-05-04 16:04 Message: Logged In: YES user_id=11105 Originator: NO FYI, this PyOpenGL bug may be related: https://sourceforge.net/tracker/?func=detail&atid=105988&aid=1712741&group_id=5988 -- Comment By: Thomas Heller (theller) Date: 2007-03-22 21:18 Message: Logged In: YES user_id=11105 Originator: NO Skip, I do not have a G5 machine available, only a G4 and the problem doesn't appear there. For someone with a G5 it should not be too difficult to reproduce (or investigate), I think that running './python Lib/test/regrtest.py test_ctypes test_sqlite' should suffice. However, if loading the GLUT library changes the cwd, and sqlite fails with a changed working dir, I'm not sure where the problem really is and how it should be fixed. It could be said that test_ctypes should restore the cwd after the tests (but maybe this should be done by the test framework?), or it could be said that test_sqlite should set the cwd in case the test relies on it. I'll assign back to you because I have no way to test it on a machine where the problem appears. -- Comment By: Skip Montanaro (montanaro) Date: 2007-03-18 18:20 Message: Logged In: YES user_id=44345 Originator: YES Thomas, I assigned this to you simply so you can cast your eye on the problem and see if you think there might be anything ctypes-related in the problem I've reported. I've ignored it up 'til now, but I bumped up against it again trying to get the Pybots SQLAlchemy test suite running on my Mac. During my previous encounter with this problem Ronald Oussouren reported on a possible connection with GLUT: http://mail.python.org/pipermail/python-dev/2006-October/069523.html Assuming there is no obvious ctypes connection you can determine (it's not clear to me that there is any current directory funny business going on) just assign it back to me and I'll try to find a pysqlite expert to take a look. Thx, Skip -- Comment By: Skip Montanaro (montanaro) Date: 2006-10-21 20:04 Message: Logged In: YES user_id=44345 Another article here: http://article.gmane.org/gmane.comp.python.devel/84487 -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1581906&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1607951 ] mailbox.Maildir re-reads directory too often
Bugs item #1607951, was opened at 2006-12-03 09:28 Message generated for change (Comment added) made by joshtriplett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1607951&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matthias Klose (doko) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox.Maildir re-reads directory too often Initial Comment: [forwarded from http://bugs.debian.org/401395] Various functions in mailbox.Maildir call self._refresh, which always re-reads the cur and new directories with os.listdir. _refresh should stat each of the two directories first to see if they changed. This cuts processing time of a series of lookups down by a factor of the number of messages in the folder, a potentially large number. -- Comment By: Josh Triplett (joshtriplett) Date: 2007-05-04 11:30 Message: Logged In: YES user_id=1785566 Originator: NO akuchling wrote: > By stat()'ing the directories, do you mean checking the mtime? I think > this isn't safe because of the limited resolution of mtime on filesystems; > ext3 seems to have a 1-second resolution for mtime, for example. Regardless of the resolution on any particular filesystem, stat and stat64 return file timestamps as time_t, which has one-second resolution. > This means that _refresh() might read a directory, and if some other process > adds or deletes a message in the same second, _refresh() couldn't detect the > change. True. mailbox.Maildir's behavior of always representing the current contents of the maildir without requiring the caller to explicitly refresh will not work with an implementation that checks for an mtime increase. The two solutions below (inotify and your suggested mtime-1 approach) would allow the automatic updates to work efficiently. > Is there some other property of directories that could be used for > a more reliable check? On Linux, you could use inotify to get a notice when anything changes in the directory. You then wouldn't even incur the overhead of a repeated directory stat, and wouldn't need to re-read the entire directory to find the new mail. > The attached patch implements checking of mtime, but I don't recommend > applying it; it causes the test suite in test_mailbox.py to break all over > the place, because the process modifies mailboxes so quickly that the mtime > check doesn't notice the process's own changes. Fair enough. > I'll wait a bit for any alternative suggestion, and then close this bug as > "won't fix". Please don't. The performance hit of repeatedly re-reading the directory makes mailbox.Maildir unusably slow on even moderately large maildirs (a few thousand messages). For the application we originally wanted mailbox.Maildir for, we had to rewrite the code to manually operate on the maildir instead, and we achieved an improvement of several orders of magnitude. akuchling: > Stray thought: would it help if the patch stored the (mtime - 1sec) instead > of the mtime? Successive calls in the same second would then always re-read > the directories, but once the clock ticks to the next second, re-reads would > only occur if the directories have actually changed. The check would be 'if > new_mtime > self._new_mtime' instead of '=='. Good idea. That would work fine as well, and I believe it would have addressed our performance problem. This would work as a good solution on platforms that lack inotify, or if you don't want to use inotify. Compared to inotify, this would have somewhat more overhead (but still far less than the current approach), and would still perform poorly if you insert messages as you go (sometimes avoidable). However, compared to the current approach, this has a massive improvement, so please consider implementing it. > Is this sort of mtime-based checking reliable on remote filesystems such as > NFS? This particular sort of checking, yes, I think so. The times do not necessarily match the system clock (because they come from a remote system that does not necessarily have a synchronized clock), but the times should remain consistent with *each other*. Thank you for looking at this problem, Josh Triplett -- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-14 11:09 Message: Logged In: YES user_id=11375 Originator: NO Stray thought: would it help if the patch stored the (mtime - 1sec) instead of the mtime? Successive calls in the same second would then always re-read the directories, but once the clock ticks to the next second, re-reads would only occur if the directories have actually changed. The check would be 'if new_mtime
[ python-Bugs-1712742 ] pprint handles depth argument incorrectly
Bugs item #1712742, was opened at 2007-05-04 09:47 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1712742&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Dmitrii Tisnek (dimaq) Assigned to: Nobody/Anonymous (nobody) Summary: pprint handles depth argument incorrectly Initial Comment: pprint.pprint( [ ], depth=0) AssertionError: depth must be > 0 pprint.pprint( [ ], depth=0.5) [[...]] pprint.pprint( [ ], depth=1) [[[...]]] I would like to see the root (that is [...]) with depth=0 (or depth=1 if you insist), either way, [[[...]]] for depth=1 is incorrect. -- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-05-04 15:17 Message: Logged In: YES user_id=984087 Originator: NO I created patch 1712742 to fix this. BTW, I think that a depth of 1 should mean the top level object. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1712742&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1710802 ] subprocess must escape redirection characters under win32
Bugs item #1710802, was opened at 2007-05-01 22:46
Message generated for change (Comment added) made by astrand
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710802&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Patrick Mézard (pmezard)
Assigned to: Peter Åstrand (astrand)
Summary: subprocess must escape redirection characters under win32
Initial Comment:
Hello,
For some reason, subprocess.Popen arguments are not processed correctly when
one of them contains a redirection character ("<>&|") when calling a batch file.
Unittest and patch are attached below. Here are the steps to reproduce it:
callee.py
"""
import sys
print sys.argv
"""
callee.bat
"""
python callee.py %*
"""
caller.py
"""
import subprocess
args = [
'ab',
'a|b',
'a&b',
]
for arg in args:
subprocess.check_call(['callee.bat', arg])
"""
Then:
"""
>python caller.py
The system cannot find the file specified.
Traceback (most recent call last):
File "caller.py", line 22, in
subprocess.check_call(['callee.bat', arg])
File "C:\Python251\lib\subprocess.py", line 461, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['callee.bat', 'a>>
"""
--
>Comment By: Peter Åstrand (astrand)
Date: 2007-05-04 23:13
Message:
Logged In: YES
user_id=344921
Originator: NO
I'm testing with a simpler pair of files:
callee.bat:
echo arg1 %1
echo arg2 %2
echo arg3 %3
caller.py:
import subprocess
subprocess.call(['callee.bat', 'ahttp://msdn2.microsoft.com/en-us/library/ms682425.aspx indicates that batch
files are handled in a special (broken?) way.
I'm reluctant to accept the attached patch. It might for this particular
situation, but as far as I can tell, we are following the documented API.
You should be able to work around this problem by quoting in your
application.
--
Comment By: Neal Norwitz (nnorwitz)
Date: 2007-05-02 08:38
Message:
Logged In: YES
user_id=33168
Originator: NO
Peter, can you take a look at this?
--
Comment By: Patrick Mézard (pmezard)
Date: 2007-05-01 22:47
Message:
Logged In: YES
user_id=1660346
Originator: YES
File Added: fix_subp
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710802&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1713252 ] character set in Japanese on Ubuntu distribution
Bugs item #1713252, was opened at 2007-05-04 22:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1713252&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Christopher Grell (cgrell) Assigned to: Nobody/Anonymous (nobody) Summary: character set in Japanese on Ubuntu distribution Initial Comment: I tried to use IDLE on new Ubuntu 7.04 and the window opens with Japanese character set -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1713252&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
