Ilya Valmianski added the comment:
As a sizing clarification, timed_dfs ~ 150GB, control_features ~30 GB,
notime_dfs ~ 2GB.
--
___
Python tracker
<https://bugs.python.org/issue38
Ilya Valmianski added the comment:
Below is the code. It segfaults with either dill or pickle on 3.6 and 3.7.
with open(output_path,'wb') as fout:
dill.dump({
'timed_dfs': timed_dfs, #large pandas
dataframe with all b
New submission from Ilya Valmianski :
Tried pickling a dictionary with multiple pandas tables and python primitive
types. Pandas tables are large so full object size is ~200GB but system should
not be OOM (crashed with ~300 GB system memory available). Reproduced on two
machines running RHEL
