[issue18171] os.path.expanduser does not use the system encoding

[Pascal Garcia]

New submission from Pascal Garcia:

The name of the user contains accents under windows.

This error occurs when using the function. expaduser("~")

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 10: 
ordinal not in range(128)

ascii is the default encoding as sys.getdefaultencoding()
If in site.py "I enable Enable the support locale" then de defaultencoding 
become cp1252 and the function works.

Expand user should use the encoding used by the system (may be 
locale.getdefaultlocale()) to decode path given by the system instead of the 
default encoding the should be the target encoding.

I do beleave some other functions may be concerned by this problem.
I detect the problem on Windows (WP and 7), but I do beleave the problem may 
happen on Linux also.

--
components: Library (Lib)
messages: 190850
nosy: plgarcia
priority: normal
severity: normal
status: open
title: os.path.expanduser does not use the system encoding
versions: Python 2.7

___
Python tracker 
<http://bugs.python.org/issue18171>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18171] os.path.expanduser does not use the system encoding

[Pascal Garcia]

Pascal Garcia added the comment:

Here are 2 logs one with the default site.py forcing defaultencoding to ascii, 
and the other to utf8.
You can see that the home dir includes accents : Pépé Not an insult to anybody 
but this stupid computer :)

When I force using the locale.getdefaultlocale() as encoding then the function 
works, but, after having called expanduser, I need to make an explicit 
decode(locale.getdefaultlocale()), or else the string can not be used to build 
path to files.

==> with ASCII

C:\Users\pépé>D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py
Traceback (most recent call last):
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py", 
line 329, in 
run()
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py", 
line 206, in run
globaldef.initDefs(wd, force)
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\globaldef.py", 
line 80, in initDefs
wrkdir= os.path.expanduser(u"~"+os.sep)
  File "C:\Python27\lib\ntpath.py", line 301, in expanduser
return userhome + path[i:]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 10: 
ordinal not in range(128)

WITH UTF8 :
C:\Users\pépé>D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py
Traceback (most recent call last):
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py", 
line 329, in 
run()
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\scolasync.py", 
line 206, in run
globaldef.initDefs(wd, force)
  File 
"D:\DevelopmentWorkspaces\SCOLASYNC\ScolaSyncNG\scolasync-ng\src\globaldef.py", 
line 80, in initDefs
wrkdir= os.path.expanduser(u"~"+os.sep)
  File "C:\Python27\lib\ntpath.py", line 301, in expanduser
return userhome + path[i:]
  File "C:\Python27\lib\encodings\utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 10: invalid 
continuation byte

--

___
Python tracker 
<http://bugs.python.org/issue18171>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18171] os.path.expanduser does not use the system encoding

[Pascal Garcia]

Pascal Garcia added the comment:

Sorry for this error.
Thanks for the solution.

Here is the code as I modify it.
wrkdir= os.path.expanduser("~"+os.sep)
loc = locale.getdefaultlocale()
if loc[1]:
encoding = loc[1]
wrkdir= wrkdir.decode(encoding)

I need to explicitally decode the string if I want to use it and have the next 
sentence working a bit further.
os.path.join(wrkdir, u"Tango\\")

Encodding is a very good motivation to go to python3, and if i didn't have 
other constraints it would be done for ages.

For this special case I think that function should return strings with the 
default encoding, and the programmer should not have to know about the 
underground to make the right decode.

But it works, thanks again.
Pascal

--
resolution: invalid -> 
status: closed -> open
type: behavior -> 

___
Python tracker 
<http://bugs.python.org/issue18171>
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com