Re: Memory leak problem

2013-09-17 Thread Brian Craft
I'm learning that the tooling for the jvm covers a spectrum from pathetically broken to non-existent. I've had some luck running jmap like "jmap -histo:live ". Pipe it through head and run with watch, and you have a crude real-time monitor. E.g. running against the processing running from a SNA

Re: Memory leak problem

2013-09-17 Thread Andy Fingerhut
If the problem persists after that change, then at least we know that it isn't the large strings produced by 'slurp' that are causing the problem, but something else. I don't have any guesses based on what you have shown where that might be. I've seen your message about difficulties using tools l

Re: Memory leak problem

2013-09-17 Thread Joachim De Beule
Dear Brian, Thanks for your reply. I tried to use jmap, unfortunately it fails: bash-3.2$ jmap -F -dump:file=heap.bin 58708 Attaching to process ID 58708, please wait... Debugger attached successfully. Server compiler detected. JVM version is 20.51-b01-457 Dumping heap to heap.bin ... Exception i

Re: Memory leak problem

2013-09-17 Thread Joachim De Beule
Dear Andy, Thanks for your reply. I am using java version 1.6.0_51. I'm a bit reluctant to upgrade however, so I was wondering how sure you are that this is indeed the problem? The problem persists after calling clojure.tools.reader.edn/read on (java.io.PushbackReader. (clojure.java.io/reader

Re: Memory leak problem

2013-09-16 Thread Brian Craft
A symptom of this would be jmap or visualvm reporting [C or char[] as the largest allocation, by class. On Monday, September 16, 2013 8:40:27 AM UTC-7, Andy Fingerhut wrote: > > Are you using a version of Java earlier than 7u6? If so, this *might* be > related to the conversation from a few da

Re: Memory leak problem

2013-09-16 Thread Andy Fingerhut
Are you using a version of Java earlier than 7u6? If so, this *might* be related to the conversation from a few days ago about functions like subs, re-find, etc. returning short strings that keep references to the longer strings they were created from: https://groups.google.com/forum/#!topic/

Memory leak problem

2013-09-16 Thread Joachim De Beule
Dear List, I'm experiencing a memory leak and I don't understand why. I have a bunch of 50 files on disk called "data-1.edn" through "data-50.edn". I perform the following code: (def all-processed-data (reduce (fn [ret f] (merge ret (process-data (clojure.tools.reader.edn/read-string (slurp f)