[ python-Bugs-1293741 ] doctest runner cannot handle non-ascii characters
Bugs item #1293741, was opened at 2005-09-17 14:41 Message generated for change (Comment added) made by akaihola You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1293741&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: GRISEL (ogrisel) Assigned to: Nobody/Anonymous (nobody) Summary: doctest runner cannot handle non-ascii characters Initial Comment: The doctest module fails when the expected result string has non-ascii charcaters even if the # -*- coding: XXX -*- line is properly set. The enclosed code sample produce the following error: Traceback (most recent call last): File "test_iso-8859-15.py", line 41, in ? _test() File "test_iso-8859-15.py", line 26, in _test tried, failed = runner.run(t) File "/usr/lib/python2.4/doctest.py", line 1376, in run return self.__run(test, compileflags, out) File "/usr/lib/python2.4/doctest.py", line 1259, in __run if check(example.want, got, self.optionflags): File "/usr/lib/python2.4/doctest.py", line 1475, in check_output if got == want: UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in range(128) -- Comment By: akaihola (akaihola) Date: 2007-05-09 11:19 Message: Logged In: YES user_id=1432932 Originator: NO I made some tests with Python 2.5 on an Ubuntu Edgy system with an UTF-8 terminal. Here's the basic test which does work correctly: # -*- encoding: utf-8 -*- __doc__ = u""" >>> print u'ä' ä """ ; import doctest ; doctest.testmod() If I start to vary the "ä" (a with umlaut) characters in "print u'ä'" (the test) and the "ä" below it (expected result), I get a UnicodeEncodeError whenever doctest tries to print a message about non-matching test output. Here's a summary of my results in the format of test | expected result | success/failure Note that \u00e4 is unicode for the "ä" character. ä | ä | success \u00e4 | ä | success ä | \u00e4 | success \u00e4 | \u00e4 | success ä | x | fails to display message x | ä | fails to display message \u00e4 | x | fails to display message x | \u00e4 | fails to display message Conclusion: test running and output checking do work correctly, but there's a problem displaying messages about non-matching output whenever either the expected output or the output produced by the test contain any extended characters. The doctest documentation doesn't give any hint on work-arounds. -- Comment By: Tim Peters (tim_one) Date: 2006-04-24 04:21 Message: Logged In: YES user_id=31435 Unassigned myself -- don't know enough about encodings. -- Comment By: Bjorn Tillenius (bjoti) Date: 2006-02-16 13:41 Message: Logged In: YES user_id=1032069 I'm quite sure that you can use non-ASCII characters in your doctest, given that it's a unicode string. So if you make your docstring a unicode string, it should work. That is: u"""Docstring containing non-ASCII characters. ... """ -- Comment By: GRISEL (ogrisel) Date: 2005-09-18 13:25 Message: Logged In: YES user_id=795041 Unfortunateny that patch does not fix my problem. The patch at bug #1080727 fixes the problem for doctests written in external reST files (testfile and DocFileTest functions). My problem is related to internal docstring encoding (testmod for instance). However, Bjorn Tillenius says: """ If one writes doctests within documentation strings of classes and functions, it's possible to use non-ASCII characters since one can specify the encoding used in the source file. """ So according to him, docstrings' doctests with non-ascii characters should work by default. So maybe my system setup is somewhat broken. Could somebody please confirm/infirm this by running the attached sample script on his system? My system config: [EMAIL PROTECTED] (on linux) python 2.4.1 with: sys.getdefaultencoding() == 'ascii' and locale.getpreferredencoding() == 'ISO-8859-15' $ file test_iso-8859-15.py test_iso-8859-15.py: ISO-8859 English text -- Comment By: Tim Peters (tim_one) Date: 2005-09-17 20:42 Message: Logged In: YES user_id=31435 Please try the patch at http://www.python.org/sf/1080727 and report back on whether it solves your problem (attaching comments to the patch report would be most useful). -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&a
[ python-Bugs-1715581 ] Const(None) in compiler.ast.Return.value
Bugs item #1715581, was opened at 2007-05-09 13:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1715581&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ali Gholami Rudi (aligrudi) Assigned to: Nobody/Anonymous (nobody) Summary: Const(None) in compiler.ast.Return.value Initial Comment: The problem happens when:: import compiler class Visitor(object): def visitReturn(self, node): print node.value source = """ def f(): return """ compiler.walk(compiler.parse(source), Visitor()) I think the value of `node.value` should have been `None` instead of `Const(None)` as it is for most other nodes when an optional part is missing. The same problem (getting `Const(None)` instead of `None`) exists for `Sliceobj.nodes[i]` and `Discard.expr`. -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1715581&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1710703 ] zipfile.ZipFile behavior inconsistent.
Bugs item #1710703, was opened at 2007-05-01 11:43
Message generated for change (Comment added) made by alanmcintyre
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710703&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Flacy (markaflacy)
Assigned to: Nobody/Anonymous (nobody)
Summary: zipfile.ZipFile behavior inconsistent.
Initial Comment:
In short, ZipFile() will not write the Central Directory entry unless you have
added a file to it. That makes it impossible to create a valid empty zip
archive.
In one of my applications, I have the need to extract a partial set of
information from one zip file and insert it into another. There are valid use
cases where the source zip archive will not have any of the files which I am
looking for. In Python 2.4, I would end up with an empty file which was
considered to be a valid empty zip archive. In Python 2.5, an empty file is
not considered a valid zip archive.
One would reasonably expect that creating a new ZipFile(mode="r") and
successfully closing it without writing any entries would result in a valid zip
archive that could be re-opened later without throwing an exception.
--
Comment By: Alan McIntyre (alanmcintyre)
Date: 2007-05-09 16:40
Message:
Logged In: YES
user_id=1115903
Originator: NO
I tried out a change to set the modified flag (_didModify) if the ZipFile
constructor ends up having mode 'w' or decides that it's appending to a
file with no existing zip structure at the end. I'm waiting on the full
regression test suite to run against it, but it passes everything in
test_zipfile.py (and I added new tests to check for the behavior with empty
files). I can post the patch if Mark hasn't had a chance to work one up
yet.
The docs don't seem to say anything about what happens if you open a
ZipFile in 'w' or 'a' and then just close it. I wouldn't mind updating the
docs to cover this if desired.
As a side note, when attempting to open an empty file in 'r' mode, a
mostly unhelpful IOError (with message "invalid parameter") gets raised in
_EndRecData when attempting to seek backwards. It seems that it would be
preferable to catch any exceptions raised by _EndRecData and raise a
BadZipFile so that it's not as cryptic.
--
Comment By: Mark Flacy (markaflacy)
Date: 2007-05-04 02:26
Message:
Logged In: YES
user_id=1344993
Originator: YES
No wonder you're confused. My description of how 2.4 worked was flat-out
wrong; empty files opened as zip files will throw IOExceptions and have
done so since 2.4 at least (I didn't look further back than that).
However, it *is* the case that 2.4 would correctly write the Central
Directory entry on zipfile close for "w" and "a" modes, even for zip files
that never had any entries written into them.
In 2.4, the ZipFile.close() method contains the line...
if self.mode in ("w", "a"): # write ending records
...while in 2.5, the test was changed to...
if self.mode in ("w", "a") and self._didModify: # write ending
records
That change was added in revision 46967 as part of the ZIP64 support and
that change breaks backwards compatibility (as well as not making a lot of
sense for the "w" case).
--
Comment By: Neal Norwitz (nnorwitz)
Date: 2007-05-02 01:06
Message:
Logged In: YES
user_id=33168
Originator: NO
Mark, can you create a patch for zipfile to make it do what you want? Do
the docs mention anything about this either way? Perhaps the docs also
need updating?
I don't know about what happened here, but I'm guessing there was some bug
fix. This change could have been intentional or not. I patch will help us
figure out what went wrong and how to proceed.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710703&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
