Paul - Clojure definitely has its benefits, but in terms of memory footprint, Java appears to be *much* more economical, unless elements can be discarded shortly after use as Christian describes, in which case it's much *less* economical.
In a Java ArrayList, only a single ArrayList object is used to store all n elements added. The objects are stored in an internal Object []. In the test, the 100,000 elements resulted in an increase of about 1,000,000 bytes for the object array, a cost of 10 bytes each. (Actually, probably less than 10, since ArrayList.ensureCapacity() increases the Object [] by 50% each time it needs increasing, so there is probably a significant amount of unused capacity in that 1,000,000 bytes.) In contrast, Clojure creates a new holder object for each item added, at a cost of 48 bytes per item for the PersistentList objects, and 52 bytes per item for the lazy cons. So the overhead of Clojure appears to be 5 times the overhead of Java, for lists that need to stay around in memory for more than a single operation. I don't mean to imply that Java is better than Clojure; in the vast majority of cases this extra memory consumption will not be a problem, and I am really enjoying working with Clojure, not only for the way functional programming is stretching my mind, but also for the productivity improvements I anticipate getting from adding this tool to my toolkit. However, I want to be objective and scientific about the relative costs of the different approaches so that decisions I make are made in an informed and objective way. The Java code I used to test this is at http://www.pastie.org/377716. - Keith On Feb 2, 1:41 pm, Paul Barry <pauljbar...@gmail.com> wrote: > Ketih, > > I think what you have done, at least at the JVM level, is create 100,000 > lists, with basically each list containing an element and a pointer to the > next element. Because one list points to the next, none of them can be > garbage collected. It seems to me that this would be roughly equivalent to > one list with 100,000 elements, in terms of memory usage. In your map > example, the memory usage is 52 * 100,000 bytes, so that's about 5MB. How > much memory is used by the equivalent Java code? > > List l = new ArrayList(); > for(int i = 0; i < 100000; i++) { > l.add("x"); > > } > > On Mon, Feb 2, 2009 at 1:06 PM, Keith Bennett <keithrbenn...@gmail.com>wrote: > > > > > All - > > > I'm curious to know how to minimize memory consumption in Clojure for > > large lists. I did the following test in repl: > > > (def d ()) > > (dotimes [_ 100000] (def d (cons "x" d))) > > > Then, I used VisualVM, an awesome free profiling tool (https:// > > visualvm.dev.java.net/) to examine the results. It indicated that the > > number of clojure.lang.PersistentList instances increased by 100,000. > > Each instance appeared to consume 48 bytes (not including the actual > > value, but only its reference, I presume). I don't think any were > > eligible for garbage collection, because I initiated gc several times, > > and they were not removed. (I know that gc() is not deterministic, > > but am pretty sure that they would have been removed. Feel free to > > correct me.) > > > Thinking that maybe the special functions like map did some magic to > > save memory, I tried this: > > > (def a (map #(* 2 %) (range 100000))) > > > The result was 100,000 clojure.lang.LazyCons objects, each of which > > consuming 52 bytes. > > > Are there alternate strategies for building a large sequence that > > would consume less memory per element? > > > Thanks, > > Keith > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
