Hi Robert, I haven't been able to dig into Clojure/JVM/GC details, so I can't speak in particular about problems e.g. with the ants application, but there are few useful conclusions that one can draw from a JVM's RSS and VIRT. It's unfortunate, but in general, Java app memory behavior can't be reliably reported on by standard unix tools. Further, for non-trivial applications, GC parameters often need to be custom-tuned to avoid pathological behavior.
In its defense, the knobs the JVM makes available are much more granular than knobs one typically has available at configuration or run time in C++. And when GC and the memory model is properly tuned, in many cases HotSpot code can run at comparable to or better efficiency than a well written C/C++ app. That said, in my experience, GC configuration is a big problem for typical Java users, and it contributes to ongoing concerns about Java. Every few weeks it seems another colleague reports service outages or frequent restarts with an app, usually a web app. These problems are usually fixable with a few changes to GC configuration. But few Java developers have familiarized themselves with the JVM's behavior at that level, and few operations folks, even those who have skills in troubleshooting application problems, are familiar in particular with JVM tuning. There are certainly still cases where Java can't compete with C++, but they're fewer and further between- quick startup and short-run apps excepted. For long running apps, HotSpot code gen is so good, even Fortran people are starting to migrate to the JVM. But as you point out, proper memory management is a key part of the solution, and in general it's still a bit of a black art. In any event, try running your JVM with -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Xloggc:/tmp/gc.out and watch or post details from the file at /tmp/gc.out for commentary. Jonah On Fri, Jun 5, 2009 at 5:18 AM, Robert Lehr<robert.l...@gmail.com> wrote: > > First, Clojure has renewed my interest in Java, specifically the JVM > and the plethora of available libraries. Clojure and Scala are the > kind of projects that inspire peoples' imaginations in a world that > they might otherwise overlook. Thank you, Rich Hickey. > > However, I have some concerns that fundamentally stem from the JVM's > performance characteristics. I hope to obtain some answers for them > that were as thoughtful as the other discussion that I read in the > group. > > I am dealing w/ some performance constraints in the current > implementation of my application. So, although Clojure offers some > features that would solve some of my problems, I am concerned about > Clojure's and the JVM's performance issues. I read the "when > performance matters" thread and, as usual, was struck by the usual > omission of memory-utilization from the discussion of the JVM's > performance characteristics. This has always puzzled me about the > Java world and the C++ vs Java debate in general. > > Simply stated, given that it stated clearly in this group that Clojure > runs, on average, slower than the JVM, is the JVM's high memory- > utilization a non-issue for the Clojure community as well? I will > explain the most recent source of my concern below. > > (I can hear people asking themselves now, "What memory-utilization > issues?") > > As part of my investigation, I spent a lot of time w/ Rich Hickey's > ant simulation, the one that he demonstrates in his video > presentations. I ran it on Linux and Windows. I tweaked the > parameters. I scrutinized the code. I observed it, both the GUI and > in "top" (on Linux) and "pmap" and "strace", etc. > > The problem was that it was not as fast as I expected it should be > given that it was using no less than 100% of the CPU on my system. > (two 3GHz Xeon CPUs [1 core]; 3GB RAM; a beefy workstation). That > this was visible in the GUI shows how slow it appeared to me. Also, > it was using 700MB RAM (VIRT in top). Sure - it was "swapped" (I'm > familiar w/ some of the interpretations of these memory issues) except > that my system has ZERO swap space. PMAP showed that 400MB of it was > heap, too, not libraries or binaries or anything else that we can > safely ignore. This was (apparently) real, allocated heap, private to > the process's address space. > > Additionally, as the simulation ran, the initial RSS of 60MB rose to > 130MB then stopped. The VIRT remained constant. I had expected that > - however I remained concerned. > > That was w/ the OpenJDK v6. So, on a whim, I installed a v5 JVM and > reran all everything and played w/ it all over again - just to see. > > Observations: it was slower - visibly. The JVM allocated less memory > - 50% less. However, at 700MB, 50% reduction is very disappointing > for a simple simulation. top showed VIRT to be 270MB which rose to > 300MB then remained constant; that differed from the v6 JVM. RSS > started at 45MB and rose to 70MB then remained constant. > > We all know the trade-off of time (CPU) for space (RAM). However, I > am baffled by the trade-offs made w/ JVM's GC, as I observed them > between the v5 and v6 JVM. Even if CPU performance was good, this > kind of memory-utilization is unacceptable for application. And that > is the part that continues to baffle me - what am I missing that so > many are willing to invest in Clojure and the JVM despite the > performance characteristics that I have observed. As I said above, > Clojure offers solutions to key problems for me. But that is > dependent on me being able to work around these performance issues. > > I apologize for the long post. When I started writing it, I didn't > expect it to be so long. I thought that the detail would be important > to understand my question sufficiently to address my question. > > I appreciate all thoughtful responses. > > -robert > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
