Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10451>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> There is also issue c) what if the filesystem encoding can only
> represent a compatibility character, say U+00B5, but not its NFKC
> equivalent, U+03BC?
It is the same problem than not being able to write U+03BC with a keyboard: in
this setup, do
STINNER Victor added the comment:
> Victor, could you please create a Reitveld review for this?
Yes, but not yet. I have first to cleanup the patch.
--
___
Python tracker
<http://bugs.python.org/iss
STINNER Victor added the comment:
> If the import lock is changed to raise an exception,
> heaps of multi-threaded software will be broken.
You are right. It has done so for 12 years already (10011), so it's a bit late
to do anything about it. And backward compatibility is very im
STINNER Victor added the comment:
> for 12 years already (10011)
Oops, it's r10011 (to get a nice URL on the commit).
--
___
Python tracker
<http://bugs.python.org
STINNER Victor added the comment:
See also issue #9260 for a possible improvment.
--
___
Python tracker
<http://bugs.python.org/issue10923>
___
___
Python-bug
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9260>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> Use "U" format to parse a module name, and "%R" to format a module name
> (to escape surrogates characters and add quotes, instead of
> "... '%.200s' ...").
See also #8754: repr() is better than str() for o
STINNER Victor added the comment:
It should be a regression introduced by #8611 or #9425.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10
Changes by STINNER Victor :
--
nosy: +georg.brandl
priority: normal -> release blocker
___
Python tracker
<http://bugs.python.org/issue10955>
___
___
Python-
STINNER Victor added the comment:
zipimport decodes filenames of the archive from cp437 or UTF-8 (depending on a
flag in each file entry). Python has a builtin UTF-8 codec, but no cp437
builtin codec. You should try to add encodings/cp437.py to your python3.2/
directory, or to build a ZIP
STINNER Victor added the comment:
Restore priority to normal: this is a workaround, and a better fix cannot be
done before 3.2 final.
--
priority: release blocker -> normal
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
The regression was introduced in r85690: use the correct encoding to decode the
filename from the ZIP file. Attached patch fixes the bootstrap issue.
--
keywords: +patch
Added file: http://bugs.python.org/file20465/issue10955.patch
STINNER Victor added the comment:
Le jeudi 20 janvier 2011 à 18:15 +, Amaury Forgeot d'Arc a écrit :
> But if the zip file contains the stdlib *and* some other custom
> modules with cp437 names, the whole operation will fail; it can be the
> case with py2exe applicatio
STINNER Victor added the comment:
> A packaging mechanism that prepares code developed on a Latin-1
> filesystem for distribution, would have to NFKC-normalize
> filenames before encoding them using UTF-8.
It causes portability issues: if you copy a non-ASCII module on a new
host, th
STINNER Victor added the comment:
Version 4 of the patch.
--
Added file: http://bugs.python.org/file20474/issue3080-4.patch
___
Python tracker
<http://bugs.python.org/issue3
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20448/issue3080-3.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
STINNER Victor added the comment:
Same patch (version 4) generated by svn.
--
Added file: http://bugs.python.org/file20475/issue3080-4-svn.patch
___
Python tracker
<http://bugs.python.org/issue3
STINNER Victor added the comment:
You can review the patch with Rietveld:
http://codereview.appspot.com/3972045
--
___
Python tracker
<http://bugs.python.org/issue3
STINNER Victor added the comment:
Oops, there is a dummy typo in imp_init_builtin() that makes test_importlib to
crash (which proves that importlib has a good coverage :-)): replace "s:" by
"U:" in if (!PyArg_ParseTuple(args, &qu
STINNER Victor added the comment:
test_reprlib fails on Windows, because '\' in quoted '\\' in the filename on
repr(module). Workaround:
***
index b0dc4d7..e476941 100644
--- a/Lib/test/test_reprlib.py
+++ b/Lib/test/test_reprlib.py
@@ -234,7 +234,7 @@ class LongReprTe
STINNER Victor added the comment:
test_runpy fails on Windows on make_legacy_pyc() (of test.support), I don't
know why.
--
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
> except for the segfault in test_importlib.
Yes, as reported in my previous comment :-) Let's update the patch for
practical reasons. But I don't want to touch
http://codereview.appspot.com/1874048 (based on patch version 4).
--
Add
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20474/issue3080-4.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
Changes by STINNER Victor :
Removed file: http://bugs.python.org/file20475/issue3080-4-svn.patch
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bug
STINNER Victor added the comment:
> No, your change is in the read_directory() function,
> which reads the whole archive the first time it's used.
Oh, I though that read_directory() only reads files one by one.
--
___
Python tra
STINNER Victor added the comment:
Ronald Oussoren and Amaury Forgeot d'Arc: do you think that it is an acceptable
limitation to only accept ASCII filenames in python32.zip? (not in all ZIP
files, just in the file loaded at startup)
All possible solutions:
a) Only accept ASCII filenam
New submission from STINNER Victor :
"./python Lib/test/regrtest.py test_zipimport_support" pass, but not with the
-R option.
$ ./python Lib/test/regrtest.py -R 3:3: test_zipimport_support
[1/1] test_zipimport_support
beginning 6 repetitions
123456
test test_zipimport_supp
STINNER Victor added the comment:
Oh, py2app is implemented in Python and use the zipfile module. So if we can
control how the filename is encoded, we can fix py2app to workaround this
limitation :-)
7zip and WinRAR uses the same algorithm than ZipFile._encodeFilename(): try
cp437 or use
New submission from STINNER Victor :
ZipInfo._encodeFilename() tries cp437 encoding or use UTF-8. It is not possible
to decide the encoding.
To workaround #10955 (bootstrap issue with python32.zip), it would be nice to
be able to create a ZIP file using only UTF-8 filenames.
Attached patch
STINNER Victor added the comment:
Oh, this patch fixes also a bug: ZipFile._RealGetContents() doesn't keep the
unicode flag, so open a ZIP file and then write it somewhere else may change
the unicode flag if unicode flag was set but the filename is also encodable to
UTF-8 (eg. ASCII fil
STINNER Victor added the comment:
#10972 has a patch for zipfile to set the filename encoding if a ZipInfo object
(to force the encoding to UTF-8).
--
___
Python tracker
<http://bugs.python.org/issue10
STINNER Victor added the comment:
7zip and WinRAR uses the same algorithm than ZipFile._encodeFilename(): try
cp437 or use UTF-8. Eg. if a filename contains ∞ (U+221E), it is encoded to
UTF-8.
WinZIP encodes all filenames to cp437: ∞ (U+221E) is replaced by 8 (U+0038), ☺
(U+263A) is
Changes by STINNER Victor :
--
title: zipfile: add unicode option to the choose filename encoding -> zipfile:
add "unicode" option to the force the filename encoding to UTF-8
___
Python tracker
<http://bugs.pytho
STINNER Victor added the comment:
On Linux, the "zip" command line program (InfoZIP zip program) only sets the
unicode flag if it is able to set the locale to "en_US.UTF-8". It can do
better: check if the locale encoding is UTF-8, and only "en_US.UTF-8" locale
STINNER Victor added the comment:
> What kind of problem are you trying to solve?
Support non-ASCII filenames in python32.zip (#10955): at bootstrap, Python 3.2
can only use UTF-8 codec (not cp437).
But I suppose also that forcing the encoding to UTF-8 gives a better Unicode
support (w
STINNER Victor added the comment:
Patch version 2: display a more useful error message:
$ python
Fatal Python error: Py_Initialize: Unable to get the locale encoding
NotImplementedError: bootstrap issue: python32.zip contains non-ASCII filenames
without the unicode flag
Aborted
Instead of
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue10973>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
> I cannot avoid haveing non-ascii filenames when a python package
> contains data files that have such names.
Are "data files" Python modules (.py files)? Or can it be anything?
--
___
Pytho
STINNER Victor added the comment:
> The question is, rather, why you need an external flag for that.
Because I don't want to change the default encoding. I am not sure that all
applications support UTF-8 encodings.
But if you control your environment, force UTF-8 encoding should impr
STINNER Victor added the comment:
No problem on Linux (Debian Sid): I tried "ŁñØ=1" in IDLE interpreter (written
using the compose key).
It looks like the bug is specific to Mac OS X and comes from Tk directly:
http://sourceforge.net/tracker/index.php?func=detail&aid=2907388&
STINNER Victor added the comment:
> I cannot start IDLE by double-clicking its icon in the Finder.
You may open a new issue for this proble.
--
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
> but I'm going to have trouble diagnosing things that don't fail
> on my development machine.
On Windows, try any character not encodable into your ANSI code page (eg. Ł
with cp1252) in the module path and non-ASCII characters in the module
STINNER Victor added the comment:
I tried issue3080-5.patch. The whole test suite pass on Windows. It pass also
on Linux with "-Wd -Werror -R 3:3:" (except #10971 which is unrelated to this
issue).
I should maybe add some unit tests for non-ASCII module paths and non-ASCII
mo
STINNER Victor added the comment:
My patch for #3080 uses repr() to format module name in all error messages.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue8
STINNER Victor added the comment:
In #10972, I propose to add an option for the filename encoding to UTF-8. But I
would like to force UTF-8 to create a ZIP file, it doesn't concern the
decompression of a ZIP file.
Proposal of a specification to fix both issues at the same
STINNER Victor added the comment:
georg.brandl> Patch #2 looks innocent enough to me,
georg.brandl> and is clearly an improvement.
Ok, issue fixed by r88140 (+r88141):
Issue #10955: zipimport uses ASCII encoding instead of cp497 to decode
filenames, at bootstrap, if the codec registry
STINNER Victor added the comment:
Extract of PEP : << Note also that strings passed to start_response() as a
status or as response headers must follow RFC 2616 with respect to encoding.
That is, they must either be ISO-8859-1 characters, or use RFC 2047 MIME
encoding. >>
STINNER Victor added the comment:
RFC 5987 (Character Set and Language Encoding for Hypertext Transfer Protocol
(HTTP) Header Field Parameters), August 2010:
http://greenbytes.de/tech/webdav/rfc5987.html#language.specification.in.encoded.words
<< 3.3 Language Specification in Encoded
New submission from STINNER Victor :
ssl.SSLContext(True).load_verify_locations(None, True) does segfault.
Py_DECREF(cafile_bytes) in Modules/_ssl.c:1686 should be replaced by
Py_XDECREF(cafile_bytes).
--
components: Library (Lib)
messages: 126901
nosy: haypo, pitrou
priority: normal
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue9124>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from STINNER Victor :
Attached script, bug.py, ends with:
$ python3.2 source2.py
deque.remove(x): x not in deque
python: ./Modules/_collectionsmodule.c:536: deque_count: Assertion
`leftblock->rightlink != ((void *)0)' failed.
Abandon (core dumped)
--
co
New submission from STINNER Victor :
Attached script (bug2.py) ends with:
$ ~/prog/SVN/py3k/python bug2.py
python: ./Modules/_threadmodule.c:399: rlock_acquire_restore: Assertion
`self
Changes by STINNER Victor :
--
versions: +Python 2.7
___
Python tracker
<http://bugs.python.org/issue11004>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
deque.patch: I'm unable to say if the patch is correct or not, but it is always
a good thing to remove asser(...) :-)
--
___
Python tracker
<http://bugs.python.org/is
Changes by STINNER Victor :
--
nosy: -haypo
___
Python tracker
<http://bugs.python.org/issue10973>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue11010>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
mailbox.patch:
- open files in binary mode not as text
- parse as bytes not as Unicode
- replace email.generator.Generator() by email.generator.BytesGenerator()
- use .message_from_bytes() instead of .message_from_str()
- use .message_from_binary_file
STINNER Victor added the comment:
While working on this issue, I found and fixed two bugs in the email binary
parser: r88196 and r88197.
--
___
Python tracker
<http://bugs.python.org/issue9
STINNER Victor added the comment:
> I reverted r88197 because it was incorrect and caused an email test
> to fail. Once I come up with a test for it I'll fix it correctly.
test_mailbox is a good (indirect) test suite for this change. The problem of
r88197 is that it replaces msg.
STINNER Victor added the comment:
All test_email and test_mailbox pass with
mailbox.patch+BytesGenerator_handle_text.patch on Windows except one test:
==
ERROR: test_set_item (test.test_mailbox.TestBabyl
STINNER Victor added the comment:
(Patch without patch from #9124)
--
keywords: +patch
nosy: +haypo
Added file: http://bugs.python.org/file20535/BytesGenerator_handle_text.patch
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
The new test fails without the fix, and it pass with the fix. The fix itself
looks correct because I wrote a similar patch :-) Go on.
--
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
pitrou> There's a missing conversion in mailbox.patch.
pitrou> Running with -bb shows the issue.
pitrou> Here is an updated patch.
Good catch: test_mailbox now pass on Windows.
--
Some remarks on mailbox2.patch.
get_string() returns a b
STINNER Victor added the comment:
> - Using locale.setlocale(..., ...)
> - Re-open causes same error, I/O layer codec has not been changed!
Yes, this is the expected behaviour with the current code.
TextIOWrapper uses indirectly locale.getpreferredencoding() to choose your file
encodi
STINNER Victor added the comment:
> This bug may be based on same problem as Issue 6203.
Nope, both issues are different. Here you want that TextIOWrapper reads your
current locale, and not your environment variables. Issue #6203 asks why
LC_CTYPE is not C by default, but the user loc
STINNER Victor added the comment:
> To add a little bit more analysis: posix.device_encoding requires that
> the LC_CTYPE is set. Setting it just in this function would not be
> possible, as setlocale is not thread-safe.
open() does indirectly (locale.getpreferredencoding()) change
STINNER Victor added the comment:
> Something is very wrong with our code too. I have dumped the text
> that's cousing the "freeze" and run it using the test case scripts.
> It worked slow, but worked.
I retried test_MIMEText.tar.bz2 on FreeBSD 8.0 with 640 MB of memor
STINNER Victor added the comment:
It looks like there is nothing interesting to do here, so I close the issue
(which is not a bug :-)).
--
resolution: -> invalid
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
Set version to 3.3, I think that it is too late to change such critical code in
Python 3.2.
--
versions: +Python 3.3 -Python 3.1, Python 3.2
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
> upon program startup, init LibC environment: setlocale(LC_ALL, "");
Python 3 does something like that: Py_InitializeEx() calls setlocale(LC_CTYPE,
""). But I (and others) consider that as a bug (see #6203 discussion): Python
shou
New submission from STINNER Victor :
$ ./python
Python 3.2rc1+ (unknown, Jan 18 2011, 00:55:20)
>>> import _string
>>> _string.formatter_field_name_split(1)
python: Objects/stringlib/string_format.h:1347: formatter_field_name_split:
Assertion `((PyObject*)(self))-&g
STINNER Victor added the comment:
Here is a patch including a test.
The test pass on regrtest with -R 3:3: (no reference leak).
--
keywords: +patch
Added file: http://bugs.python.org/file20563/ssl.patch
___
Python tracker
<http://bugs.python.
STINNER Victor added the comment:
Should it be fixed in Python 3.2 or not? (load_verify_locations was introduced
with SSLContext in Python 3.2)
--
priority: high -> release blocker
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
Ok, fixed in r88228. Not backport needed, SSLContext was introduced in Python
3.2.
--
resolution: accepted -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/i
STINNER Victor added the comment:
+if isinstance(message, io.TextIOWrapper):
+# Backward compatibility hack.
+message = message.buffer
Is it a good thing to parse a mailbox using a text file? If not, we should emit
a warning and maybe remove this
STINNER Victor added the comment:
> The last step is running the tests on Windows.
> Attached is the updated patch.
mailbox4.patch doesn't pass on Windows, Raymond is working on a patch.
--
___
Python tracker
<http://bugs.python
STINNER Victor added the comment:
As explained in issue #10828: Python 3.2 doesn't support non-ASCII module names
on Windows because module names are encoded to UTF-8 instead of the filesystem
encoding (the ANSI code page).
--
___
Python tr
STINNER Victor added the comment:
"Python’s import mechanism can now load modules installed in directories with
non-ASCII characters in the path name: import møøse.bites"
møøse is not a module *path*, but a module *name*... This example doesn't work
on Windows: see #3080. Modu
STINNER Victor added the comment:
Should we add imports in all examples? Eg. add import math in:
>>> repr(math.pi)
'3.141592653589793'
>>> str(math.pi)
'3.141592653589793'
At least, accumulate should be replaced by itertools.accumulate in the
following
STINNER Victor added the comment:
The import in the following example is wrong :
>>> import datetime
>>> datetime.now(timezone.utc)
...
It should be replaced by: from datetime import datetime, timezone.
--
___
Pyt
STINNER Victor added the comment:
ABCMeta should be replaced by abc.ABCMeta, or other "abc." prefixes should be
removed.
class Temperature(metaclass=ABCMeta):
@abc.abstractclassmethod
def from_fahrenheit(self, t):
...
@abc.abstractclassmethod
def from_celsi
Changes by STINNER Victor :
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue7358>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
If you are still able to reproduce the bug, you may try the following module to
get a backtrace: https://github.com/haypo/faulthandler/
--
status: pending -> open
___
Python tracker
<http://bugs.pyth
STINNER Victor added the comment:
See also #6011.
--
___
Python tracker
<http://bugs.python.org/issue3080>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
title: ZipFile and CP932 encoding -> ZipFile: add a filename_encoding argument
___
Python tracker
<http://bugs.python.org/issu
STINNER Victor added the comment:
Attached patch replaces locale.getpreferredencoding() by
locale.getpreferredencoding(False) in _io.TextIOWrapper and _pyio.TextIOWrapper.
--
keywords: +patch
Added file: http://bugs.python.org/file20637/io_dont_set_locale.patch
STINNER Victor added the comment:
I opened other tickets related to PyUnicode_FromFormatV:
* #10833 :Replace %.100s by %s in PyErr_Format(): the arbitrary limit of 500
bytes is outdated
* #10831: PyUnicode_FromFormatV() doesn't support %li, %lli, %zi
* #10830: PyUnicode_FromFormat
STINNER Victor added the comment:
You can add it to MANIFEST.in.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue11092>
___
___
Python-bugs-list m
STINNER Victor added the comment:
Since r70638, the http client encodes unicode to ISO-8859-1:
<< RFC 2616 says that iso-8859-1 is the default charset for HTTP entity
bodies, but we encoded strings using ascii. See
http://bugs.python.org/issue5314. Changed docs and code to use
iso-
STINNER Victor added the comment:
Since r7409 (14 years ago), Python does set the binary binary mode on stdin and
stdout (not stderr) if the -u flag is used:
if (unbuffered) {
#if defined(MS_WINDOWS) || defined(__CYGWIN__)
_setmode(fileno(stdin), O_BINARY);
_setmode
STINNER Victor added the comment:
> Since 2.x is closed for new features, this has to be rejected.
We can explain in ElementTree documentation how to pass non-ASCII unicode
strings: using explicit encoding to UTF-8.
--
nosy: +haypo
status: pending ->
STINNER Victor added the comment:
gcc 4.6 bug has been fixed (http://gcc.gnu.org/bugzilla/show_bug.cgi?id=47271).
So setup.py can compile extensions using gcc 4.6, and here are new warnings:
---
building dbm using bdb
/home/haypo/prog/GIT/py3k/Modules/_pickle.c: In function 'load':
/
STINNER Victor added the comment:
Can you try to get a backtrace? Use make install SHELL="bash -x" to display
executed shell commands. And then rerun the last command in gdb: after the
crash, the "where" command will give you a backtrace.
---
Changes by STINNER Victor :
--
nosy: +r.david.murray
___
Python tracker
<http://bugs.python.org/issue6>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by STINNER Victor :
--
nosy: +brett.cannon
___
Python tracker
<http://bugs.python.org/issue10971>
___
___
Python-bugs-list mailing list
Unsubscribe:
STINNER Victor added the comment:
You should maybe add a test into _testcapi for this issue.
--
nosy: +haypo
___
Python tracker
<http://bugs.python.org/issue11
STINNER Victor added the comment:
Ah, there is already a test for that: ok, it's fine and enough.
--
___
Python tracker
<http://bugs.python.org/is
New submission from STINNER Victor :
If the script filename is not decodable from the filesystem encoding, Python
fails with a UnicodeEncodeError when we reach the recursion limit. The problem
doesn't come from the user script, but from Python internals. It is difficult
to understand an
New submission from STINNER Victor :
The compileall uses print("bla", filename, "bla") to write messages to the
console. But the print fails if the filename cannot be encoded to the console
encoding. It occurs if the filename is an undecodable filename encoded by the
PEP
3501 - 3600 of 35168 matches
Mail list logo
