New submission from Pete Wicken <petewic...@gmail.com>:
Originally found as an issue in Lib/shelve.py; if we attempt to pickle a
builtin as the program is exiting then Modules/_pickle.c will fail at the point
of the PyImport_Import in save_global. In CPython3.8 this causes a segfault, in
CPython3.9 a PicklingError is raised.
This is especially problematic in shelve.py as object pickling is attempted by
the __del__ method's call stack when writeback=True. Therefore if the program
exits before an explicit sync is called; in 3.8 the data will not be written to
disk and a segfault occurs; in 3.9 the data is written to disk, but with an
uncaught exception being raised.
Exception demonstrated via shelve on 3.9.1 on MacOS with Clang:
Python 3.9.1 (default, Dec 10 2020, 11:11:14)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
Exception ignored in: <function Shelf.__del__ at 0x10ed67790>
Traceback (most recent call last):
File ".../3.9/lib/python3.9/shelve.py", line 162, in __del__
File ".../3.9/lib/python3.9/shelve.py", line 144, in close
File ".../3.9/lib/python3.9/shelve.py", line 168, in sync
File ".../3.9/lib/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module
'builtins' failed
Segfault demonstrated via shelve on 3.8.5 on MacOS with Clang (different system
from above):
Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open('testing', writeback=True)
>>> s['a'] = Exception
>>> exit()
[1] 10040 segmentation fault python3.8
Exception demonstrated via shelve on 3.9.1 on RHEL with GCC:
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shelve
>>> s = shelve.open("thing", writeback=True)
>>> s['a'] = Exception
>>>
Exception ignored in: <function Shelf.__del__ at 0x7f3f30b7c820>
Traceback (most recent call last):
File "/usr/lib64/python3.9/shelve.py", line 162, in __del__
File "/usr/lib64/python3.9/shelve.py", line 144, in close
File "/usr/lib64/python3.9/shelve.py", line 168, in sync
File "/usr/lib64/python3.9/shelve.py", line 124, in __setitem__
_pickle.PicklingError: Can't pickle <class 'Exception'>: import of module
'builtins' failed
Code example to reproduce using Pickle in the class __del__, demonstrated on a
RHEL system:
Python 3.9.1 (default, Dec 8 2020, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pickle import DEFAULT_PROTOCOL, Pickler
>>> from io import BytesIO
>>> class T:
... def __del__(self):
... f = BytesIO()
... p = Pickler(f, DEFAULT_PROTOCOL)
... p.dump(sum)
...
>>> t = T()
>>> exit()
Exception ignored in: <function T.__del__ at 0x7f5f04d9ef70>
Traceback (most recent call last):
File "<stdin>", line 5, in __del__
_pickle.PicklingError: Can't pickle <built-in function sum>: import of module
'builtins' failed
Have not tested on 3.6, 3.7 or 3.10.
----------
components: Library (Lib)
messages: 385110
nosy: Wicken
priority: normal
severity: normal
status: open
title: Pickle can't import builtins at exit
type: crash
versions: Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue42935>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
