New submission from papad <dimapsh...@gmail.com>:
There seems to be a race condition when unpickling the same object from
different threads (My guess is it's related to imports that are related to the
unpickled object).
I have used the following files for reproduction (File contents are in the
attached archive, also pasted at the bottom of the issue report):
import_me.py - Containing the object we will pickle
pickle_object.py - the code to pickle the object
trigger.py - a piece of code to unpickle the object from different threads
simultaneously and trigger the crash
I have used the files in the attached archive (containing the 3 files mentioned
above)
to reproduce run the following commands:
pickle_object.py
trigger.py
Running trigger.py will crash on about 50% of the runs with the following error:
Traceback (most recent call last):
File "./trigger.py", line 16, in pickle_load_thread
pickle.load(h)
AttributeError: Can't get attribute 'PickleMe' on <module 'import_me' from
'/data/import_me.py'>
I have tested this on the following software stacks:
1. python:3.7.3 docker
- linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC
2019 x86_64 GNU/Linux
- python: 3.7.3
- distro: Debian GNU/Linux 9 (stretch)
2. my laptop
- linux version: 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 08:28:49 UTC
2019 x86_64 x86_64 x86_64 GNU/Linux
- python: 3.6.7
- distro: Ubuntu 18.04.2 LTS
I'm uncertain if this is a bug related to pickle or python module importing.
Similar issues I've found: https://bugs.python.org/issue12680 and
https://bugs.python.org/issue30891.
---------------------------------------------------------------------------------------
filename: import_me.py
#! /usr/bin/python3
class PickleMe(object):
def __init__(self):
self.a = "a"
---------------------------------------------------------------------------------------
filename: pickle_object.py
#! /usr/bin/python3
import pickle
from import_me import PickleMe
p = PickleMe()
with open('pickled', 'wb') as h:
pickle.dump(p, h)
---------------------------------------------------------------------------------------
filename: trigger.py
#! /usr/bin/python3
import threading
import pickle
import logging
threads = []
def pickle_load_thread():
logging.error("Thread %d loading", threading.get_ident())
try:
with open('pickled', 'rb') as h:
pickle.load(h)
except:
logging.exception("Exception in loading")
def start_pickle_load_thread():
for x in range(2):
load_thread = threading.Thread(target=pickle_load_thread)
load_thread.daemon = True
threads.append(load_thread)
for x in threads:
x.start()
if __name__ == '__main__':
start_pickle_load_thread()
for t in threads:
t.join()
----------
components: Library (Lib)
files: reproduce.tar.gz
messages: 341257
nosy: papad
priority: normal
severity: normal
status: open
title: Race condition during pickle.load()
type: crash
versions: Python 3.7
Added file: https://bugs.python.org/file48294/reproduce.tar.gz
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue36773>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
