Dear All,
while working with some big dictionaries I realized that save consumes a lot
of RAM. Here is an example.
Suppose foo is a dictionary whose keys are integer polynomials in three
variables and whose values are integers. In my running example it has 342971
entries and it occupies 10 Mb in memory:
sage: import sys
sage: sys.getsizeof(foo)
10485848
Before saving the file this is what top has to say about my running sage:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13099 VulK 20 0 19.1g 1.8g 84988 R 1.0 2.8 191:55.58 python3
/opt/sage/src/bin/sage-ipython -i
And here is a snapshot I got moments before saving ended
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13099 VulK 20 0 32.6g 15.2g 86140 R 100.3 24.2 197:58.38 python3
/opt/sage/src/bin/sage-ipython -i
I am using the following instruction:
sage: save(foo,"/tmp/bar")
A little poking around shows that, for object without a save method, save
calls _base_save which in turn calls _save_dump
sage: from sage.misc.persist import _base_dumps
sage: baz = _base_dumps(bar, compress=True)
sage: sys.getsizeof(baz)
175066342
baz is 17 times bigger than foo but it still reasonable: is it abot 170Mb;
this figure agrees with the size of the savefile. What seems unreasonable is
that to produce it sage used roughly 13Gb of memory.
Is this the expected behaviour? Can anyone provide me with some insights on
how pickling works in sage?
Thanks
S.
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sage-devel/Zqyrg_z9CEwhgRG2%40bunion.
