Christian Tanzer added the comment:
> The code works when using encoding='bytes'. Thanks Tim for the suggestion.
> So this is not a bug, but is there any sense in having encoding='ASCII' by
> default in pickle ?
It is most definitely a bug. And it adds another road block to moving python
applications from 2.7 to 3.x!
encoding='bytes' has serious side effects and isn't useful in the general case.
For instance, it will result in dict-keys being unpickled as bytes instead of
as str after which hilarity ensues.
I got the exception
UnicodeDecodeError: 'ascii' codec can't decode byte 0xdf in position 1:
ordinal not in range(128)
when testing an application for compatibility in Python 3.5 on a pickle created
by Python 2.7. The pickled data is a nested data structure and it took me quite
a while to determine that the single datetime instance was the culprit.
Here is a small test case that reproduces the problem::
# -*- coding: utf-8 -*-
# pickle_dump.py
import datetime, pickle, uuid
dti = datetime.datetime(2015, 10, 12, 13, 17, 42, 123456)
data = { "ascii" : "abc", "text" : u"äbc", "int" : 42, "date-time" : dti }
with open("/tmp/pickle.test", "wb") as file :
pickle.dump(data, file, protocol=2)
# pickle_load.py
# -*- coding: utf-8 -*-
import pickle
with open("/tmp/pickle.test", "rb") as file :
data = pickle.load(file)
print(data)
$ python2.7 pickle_dump.py
$ python2.7 pickle_load.py
{'ascii': 'abc', 'text': u'\xe4bc', 'int': 42, 'date-time':
datetime.datetime(2015, 10, 12, 13, 17, 42, 123456)}
$ python3.5 pickle_load.py
Traceback (most recent call last):
File "pickle_load.py", line 6, in <module>
data = pickle.load(file)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xdf in position 1: ordinal
not in range(128)
That error message is spectacularly useless.
----------
nosy: +tan...@swing.co.at
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue22005>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
