A symptom of this would be jmap or visualvm reporting [C or char[] as the largest allocation, by class.
On Monday, September 16, 2013 8:40:27 AM UTC-7, Andy Fingerhut wrote: > > Are you using a version of Java earlier than 7u6? If so, this *might* be > related to the conversation from a few days ago about functions like subs, > re-find, etc. returning short strings that keep references to the longer > strings they were created from: > > https://groups.google.com/forum/#!topic/clojure/PeHTCWNgrL0 > > Even if you are using such a Java version, I have not confirmed whether > the function clojure.tools.reader.edn/read-string returns substrings of the > input string. > > One quick thing to try that might help is to call > clojure.tools.reader.edn/read on the File object f directly, without > calling slurp. That will at least avoid the behavior of creating big > strings with the entire contents of each file. It should also avoid the > "small substrings holding references to large original strings", if that is > indeed what you are experiencing. > > Andy > > > On Mon, Sep 16, 2013 at 8:17 AM, Joachim De Beule > <joachim....@gmail.com<javascript:> > > wrote: > >> Dear List, >> >> I'm experiencing a memory leak and I don't understand why. >> >> I have a bunch of 50 files on disk called "data-1.edn" through >> "data-50.edn". I perform the following code: >> >> (def all-processed-data (reduce (fn [ret f] (merge ret (process-data >> (clojure.tools.reader.edn/read-string (slurp f))))) {} file-list)) >> >> "file-list" is a sequence of java.io.File objects pointing towards the 50 >> data files. >> "process-data" is a function that produces a map from the data read. >> >> Although each individual datafile is about 100 Megabytes, the result of >> processing a datafile is a map of about 200k only, so that the map returned >> by the entire above expression is only about 10 Megabytes. Nevertheless, >> executing the above code fills up memory and eventually stops clojure from >> functioning. Why? >> >> Thanks a lot for any suggestions! >> >> Joachim. >> >> -- >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com<javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
