Re: Non-unicode file names

2018-08-09 Thread Thomas Jollans
On 09/08/18 05:13, INADA Naoki wrote: > Please use Python 3.7. > > Python 3.7 has several improvements on this area. Thanks! Darkly remembering something about UTF-8 mode, I suspected it might... > > * When PEP 538 or 540 is used, default error handler for stdio is > surrogateescape > * You can

Re: Non-unicode file names

2018-08-08 Thread Marko Rauhamaa
INADA Naoki : > For Python 3.6, I think best way to allow arbitrary bytes on stdout is > using `PYTHONIOENCODING=utf-8:surrogateescape` environment variable. Good info! Marko -- https://mail.python.org/mailman/listinfo/python-list

Re: Non-unicode file names

2018-08-08 Thread INADA Naoki
Please use Python 3.7. Python 3.7 has several improvements on this area. * When PEP 538 or 540 is used, default error handler for stdio is surrogateescape * You can sys.stdout.reconfigure(errors='surrogateescape') For Python 3.6, I think best way to allow arbitrary bytes on stdout is using `PYTH

Re: Non-unicode file names

2018-08-08 Thread Cameron Simpson
On 09Aug2018 03:14, MRAB wrote: [...] Is it true that Unix filenames can contain control characters, e.g. \x07? Yep. They're just byte strings. You can't have \0 (NUL) because the API uses NUL terminated strings, and you can't use slash '/' in the filename components because that is the comp

Re: Non-unicode file names

2018-08-08 Thread MRAB
On 2018-08-09 01:14, Thomas Jollans wrote: On 09/08/18 01:48, MRAB wrote: On 2018-08-08 23:16, Thomas Jollans wrote: On *nix, file names are bytes. In real life, we prefer to think of file names as strings. How non-ASCII file names are created is determined by the locale, and on most systems th

Re: Non-unicode file names

2018-08-08 Thread Thomas Jollans
On 09/08/18 01:48, MRAB wrote: > On 2018-08-08 23:16, Thomas Jollans wrote: >> On *nix, file names are bytes. In real life, we prefer to think of file >> names as strings. How non-ASCII file names are created is determined by >> the locale, and on most systems these days, every locale uses UTF-8 an

Re: Non-unicode file names

2018-08-08 Thread MRAB
On 2018-08-08 23:16, Thomas Jollans wrote: On *nix, file names are bytes. In real life, we prefer to think of file names as strings. How non-ASCII file names are created is determined by the locale, and on most systems these days, every locale uses UTF-8 and everybody's happy. Of course this does

Non-unicode file names

2018-08-08 Thread Thomas Jollans
On *nix, file names are bytes. In real life, we prefer to think of file names as strings. How non-ASCII file names are created is determined by the locale, and on most systems these days, every locale uses UTF-8 and everybody's happy. Of course this doesn't mean you'll never run into and old direct

Re: Using Unicode file names with ftplib and encodings.idna as a workaround?

2010-06-22 Thread Gabriel Genellina
En Tue, 22 Jun 2010 11:43:34 -0300, escribió: Python 2.6.5 (Win32): Is there a work around for ftplib's (and ftputil's) apparent inability to support Unicode file names? I'm thinking that I might be able to use the encodings.idna as a work around for this? According to RFC 2

Using Unicode file names with ftplib and encodings.idna as a workaround?

2010-06-22 Thread python
Python 2.6.5 (Win32): Is there a work around for ftplib's (and ftputil's) apparent inability to support Unicode file names? I'm thinking that I might be able to use the encodings.idna as a work around for this? In other words, regardless of whether I'm getting or putting a f

Re: WinXP, Python3.1.2, dir-listing to XML - problem with unicode file names

2010-04-03 Thread Mark Tolonen
"kai_nerda" wrote in message news:hp69ri+a...@egroups.com... Hi, OS = Windows XP (German language) Python = 3.1.2 I need to write a directory listing into a XML file. And after hours of trying and searching i have no clue. My main problem is that the file and folder names can have characters

WinXP, Python3.1.2, dir-listing to XML - problem with unicode file names

2010-04-03 Thread kai_nerda
Hi, OS = Windows XP (German language) Python = 3.1.2 I need to write a directory listing into a XML file. And after hours of trying and searching i have no clue. My main problem is that the file and folder names can have characters of different languages like German, Turkish, Russian, maybe else

Re: Unicode File Names

2008-10-18 Thread Mark Tolonen
""Martin v. Löwis"" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] Oddly, os.getcwd() and os.getcwdu() both still exist in Python 3.0. Since the behavior is now identical it seems os.getcwdu() should be dropped. It is dropped, and os.getcwdb() has been added. Must be changed po

Re: Unicode File Names

2008-10-18 Thread Martin v. Löwis
> Oddly, os.getcwd() and os.getcwdu() both still exist in Python 3.0. > Since the behavior is now identical it seems os.getcwdu() should be > dropped. It is dropped, and os.getcwdb() has been added. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list

Re: Unicode File Names

2008-10-18 Thread Mark Tolonen
"Jordan" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] l = os.listdir(unicode(os.getcwd())) Other options to get the same result: l = os.listdir(os.getcwdu()) l = os.listdir(u'.') Oddly, os.getcwd() and os.getcwdu() both still exist in Python 3.0. Since the behavior is now

Re: Unicode File Names

2008-10-18 Thread John Machin
On Oct 18, 5:57 pm, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > Should the note be removed, or should it say something like "Unicode > > file names are supported. New in Python 2.6."? Is there anything else > > that should be mentioned? >

Re: Unicode File Names

2008-10-18 Thread Martin v. Löwis
> Should the note be removed, or should it say something like "Unicode > file names are supported. New in Python 2.6."? Is there anything else > that should be mentioned? The note should be corrected, documenting the behaviour implemented. > More on cp437: I see where you

Re: Unicode File Names

2008-10-17 Thread John Machin
d astray by the fact that the 2.6 docs still contain the note that the OP asked about: "There is no official file name encoding for ZIP files. If you have unicode file names, you must convert them to byte strings in your desired encoding before passing them to write(). WinZip interprets all

Re: Unicode File Names

2008-10-17 Thread Martin v. Lo
> Step 4: Either wait for Python 2.7 or apply the patch to your own copy > of zipfile ... Actually, this is released in Python 2.6, see r62724. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list

Re: Unicode File Names

2008-10-16 Thread John Machin
rs in > position 13-16: character maps to > > I'm perfectly willing to let command prompt refuse to print that (it's > debugging only) if the next issue was resolved >_>: use print repr(thing) for debugging. > > """ > Note: There is no o

Re: Unicode File Names

2008-10-16 Thread Jordan
g' u'03-\u3072\u3089\u304c\u306a.jpg' >>>for thing in l: ...print thing 01-ひらがな.jpg 02-ひらがな.jpg 03-ひらがな.jpg [/code] Yay. Having a file that tries "for thing in l: print thing" fails with: File "C:\Python25\Lib\encodings\cp437.py", line 12, in encode

Re: Unicode File Names

2008-10-16 Thread Jordan
On Oct 16, 10:18 pm, John Machin <[EMAIL PROTECTED]> wrote: > On Oct 17, 12:52 pm, Jordan <[EMAIL PROTECTED]> wrote: > > > > > On Oct 16, 9:20 pm, John Machin <[EMAIL PROTECTED]> wrote: > > > > On Oct 17, 11:43 am, Jordan <[EMAIL PROTECTED]> wrote: > > > > > I've got a bunch of files with Japanese

Re: Unicode File Names

2008-10-16 Thread John Machin
On Oct 17, 12:52 pm, Jordan <[EMAIL PROTECTED]> wrote: > On Oct 16, 9:20 pm, John Machin <[EMAIL PROTECTED]> wrote: > > > > > On Oct 17, 11:43 am, Jordan <[EMAIL PROTECTED]> wrote: > > > > I've got a bunch of files with Japanese characters in their names and > > > os.listdir() replaces those charac

Re: Unicode File Names

2008-10-16 Thread Chris Rebert
2008/10/16 Jordan <[EMAIL PROTECTED]>: > On Oct 16, 9:20 pm, John Machin <[EMAIL PROTECTED]> wrote: >> On Oct 17, 11:43 am, Jordan <[EMAIL PROTECTED]> wrote: >> >> > I've got a bunch of files with Japanese characters in their names and >> > os.listdir() replaces those characters with ?'s. I'm tryin

Re: Unicode File Names

2008-10-16 Thread Jordan
On Oct 16, 9:20 pm, John Machin <[EMAIL PROTECTED]> wrote: > On Oct 17, 11:43 am, Jordan <[EMAIL PROTECTED]> wrote: > > > I've got a bunch of files with Japanese characters in their names and > > os.listdir() replaces those characters with ?'s. I'm trying to open > > the files several steps later,

Re: Unicode File Names

2008-10-16 Thread Seun Osewa
Try Python 3. Python 3 strings are native by default, so the os.listdir() in Python 3 should support the Japanese characters. On Oct 17, 1:43 am, Jordan <[EMAIL PROTECTED]> wrote: > I've got a bunch of files with Japanese characters in their names and > os.listdir() replaces those characters with

Re: Unicode File Names

2008-10-16 Thread John Machin
On Oct 17, 11:43 am, Jordan <[EMAIL PROTECTED]> wrote: > I've got a bunch of files with Japanese characters in their names and > os.listdir() replaces those characters with ?'s. I'm trying to open > the files several steps later, and obviously Python isn't going to > find '01-.jpg' (formally '0

Unicode File Names

2008-10-16 Thread Jordan
I've got a bunch of files with Japanese characters in their names and os.listdir() replaces those characters with ?'s. I'm trying to open the files several steps later, and obviously Python isn't going to find '01-.jpg' (formally '01-ひらがな.jpg') because it doesn't exist. I'm not sure where in th