Freja Nordsiek <fnord...@gmail.com>: > If I was to hazard a reason for why Guile gets very slow when loading > 20 GB or more (may or may not be related to it being buggy and > crashy), my guesses would be a lot of the data when loaded into Guile > was allocated such that the GC scans it for pointers (using > scm_gc_malloc instead of scm_gc_malloc_pointerless) which would vastly > increase the amount of memory the GC needs to scan every time it runs.
Good point! If you didn't to any C programming, what kind of native Guile data structures are good for such large random-access storage? At least arrays haven't specifically been documented for such GC optimization: <URL: https://www.gnu.org/software/guile/manual/html_node/Arrays.htm l#Arrays>. Maybe bytevectors would do: <URL: https://www.gnu.org/software/guile/m anual/html_node/Bytevectors.html#Bytevectors>. Of course constantly encoding to and decoding from a bytevector using scheme code might be very slow without the help of some binary bulk formatting facilities for the data records. Marko
