New submission from mapf :
I have a program where I create some relatively nested data and within the same
session, I have no issues accessing the data. I then use picke.dump() with
pickle.HIGHEST_PROTOCOL to save the data so I can access it in a later session.
These files are usually over 2GB large since they contain many images in the
form of numpy arrays and I have never had any issues loading them. However
there is one data structure that is a structured numpy array of type "a" with
currently 16 different dtypes and they can all be accessed in the same session
where they were created without any problems sometimes even after dumping and
loading the data again. They can also all be accessed after they have been
loaded in a different session with the exeption of one field.
This field contains rather nested data which is why I thought that this might
be the issue, but I have honestly no idea.
Each entry in this field is a list of len 20, whose entries are either None or
a 1-d slice of "()"-shape from another structured array of type "b". This slice
in turn has 37 different dtypes, most of which are either int, fload or bool.
But there is one entry which is a list that can contain several dicts. The
entries of this dict are floats, however one can be a slice of type "b" again,
so there is some cross-referencing going on. As a test I already removed this
entry though and it still crashed.
My point is, the data that is stored is not of some crazy custom type. All the
data is either of type bool, int, fload, list, dict or numpy.array. As I said,
ALL the other stored data can be accessed without any problems. It is only this
one field that can only be accessed during the same session it was created.
My program runs using a PyQt5 GUI and I use PyCharm as the editor. I have
already read that in the past, these two in combination seem to cause this
error rather frequently maybe that has something to do with it.
I have already tried reinstalling my Python distribution as well as PyCharm as
well as running the code on a different machine to no avail.
I am also pretty certain that this used to work just last week ago. I didn't
change my code but now it doesn't work anymore.
Relevant specs:
Windows 10 Home 64 bit
PyCharm 2019.3.1 Professional
Python 3.7.4 via Anaconda
Numpy 1.16.5
PyQt 5.9.2
--
messages: 360481
nosy: mapf
priority: normal
severity: normal
status: open
title: Process finished with exit code -1073741819 (0xC005) when trying to
access data from a pickled file
type: crash
versions: Python 3.7
___
Python tracker
<https://bugs.python.org/issue39423>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
