This is the best summary of how Clojure *could* benefit from invokedynamic.
Clojure doesn't dispatch as dynamically as JRuby, but there is a dynamic component...specifically, you have to go get the fn. That repeated "get" would disappear with invokedynamic, In fact, anywhere you're going after something that's immutable but requires a memory access, you could potentially use invokedynamic to eliminate repeat memory accesses. I assume that would help in many places. Also, as Paul notes below, there's no need for multiple-arity interfaces to get full-speed perf if you use invokedynamic; JRuby's invokedynamic-based dispatch ends up going directly from call site to target code body, with no intervening interface at all. Listen to Paul. He's right. - Charlie On Friday, August 26, 2011 5:37:53 AM UTC-5, Paul Stadig wrote: > > On Thu, Aug 25, 2011 at 5:41 PM, Tal Liron <tal....@gmail.com> wrote: > >> So, after setting up a JVM 7 environment to play with Clojure, and >> enthusiastically rummaging through the codebase, I have good news and bad >> news. :) >> >> So, when Clojure calls a function, it either already has the instance in >> its entirety (a lambda) or it finds it by dereferencing a binding. Since all >> functions are instances that implement IFn, the implementation can then call >> invokeinterface, which is very efficient. >> >> [See clojure.lang.Compiler#InvokeExpr.emitArgsAndCall] >> > > What I said in another post was that with invokedynamic (assuming you could > just go whole hog into Java 7) you don't need the IFn interface. You just > need a MethodHandle. So yes, I agree that given all the existing Clojure > code, and the desire to stay compatible with Java 5 or 6 or whatever, > invokedynamic isn't very compelling. If you were going to do a green field > project, and didn't mind the Java 7 requirement, invokedynamic is very > compelling, because you can basically push a bunch of code down into the > JVM. This mean much less work, and it means you get to benefit from any > improvements and optimizations that get built into the JVM. Even for Clojure > having an optional Java 7 target may still be interesting in that it allows > you to hook into all the work that will be done on invokedynamic to support > the whole dynamic language ecosystem on the JVM. > > >> Clojure can get away with this especially easily because the only variant >> for function signatures in Clojure is arity. So, all we need is a simple >> switch to call the IFn method with the correct arity. (Interestingly, this >> means that Clojure has a fixed set of such signatures, and thus a fixed >> upper limit to arity: 20, in case you were wondering.) >> >> In a language like Ruby, methods are not so free floating, and have much >> more complex invocation styles as well as flexible signatures. (Clojure has >> only one style: function calls are simple forms.) So, in order to use >> invokeinterface, JRuby implementors would have had to create special classes >> *per* method *per* invocation style if they wanted to be efficient. But this >> is impossible, because so many classes would exhaust the perm-gen. For those >> languages, invokedynamic is a terrific solution, because it lets them >> dynamically link the implementation to the caller via a flexible >> "bootstrapping" mechanism, allowing them to do entirely without the extra >> class definition. Since all Clojure functions share essentially the same >> class as well as interface, none of these challenges exist. >> > > It's very possible that I'm missing something, but I'm not quite sure what > you're getting at here. In JRuby, for instance, the method overloading is > the same as in Clojure, it is based on arity. The slowdown is in dispatch > where you have to check a singleton class, your class, your ancestors, then > method_missing, etc. Once you've looked this all up you cache it to speed it > up later. Then you want to invalidate the cache when the world changes. > > Clojure has similar needs. For instance, Clojure has protocol dispatch and > multimethod dispatch as well as Var based dispatch. The multimethod dispatch > does a look up in the derivation hierarchy. All of this stuff can change at > runtime, just like in Ruby you can add methods to a class, change methods on > a class, etc. I don't see how Clojure's dispatch needs are much different > than what other dynamic languages are doing. > > >> Another aspect is the immutability of these IFn instances: you can't >> refactor them at runtime, all you can do is change the bindings to refer to >> new functions. So, Clojure achieves runtime dynamics by letting you simply >> rebind new functions to existing Vars, Refs, Atoms, etc., and the same >> invocation route continues as usual. In a language like Ruby, the >> bootstrapping mechanism of invokedynamic lets implementors change the base >> linkage to reflect a new signature for the method. Again, a terrific JVM 7 >> feature that Clojure simply does not need. >> > > The "signature" of a JRuby method is not much different than a Clojure > method. It is an arity overloaded method with generic arguments. They do > similar things to lots of other dynamic languages (including Clojure) like > generating a class with a bunch of different versions of the same method > with different numbers of generic arguments ( > http://jruby.org/git?p=jruby.git;a=blob;f=src/org/jruby/internal/runtime/methods/DynamicMethod.java;h=b65854c057c8a1acec57d6dad95158f08c960dcc;hb=HEAD#l203). > > invokedynamic isn't about method signatures it's about dynamic dispatch, > which is what Clojure does. > > >> Another issue I examined was how Clojure calls non-Clojure JVM code, >> thinking that perhaps there invokedynamic would be useful. But, Clojure >> again has a straightforward approach that poses no problems. Here, Clojure >> very directly calls non-Clojure code using invokestatic, invokevirtual or >> invokeinterface as appropriate. A Clojure form with the "." or ".." >> notations translates quite directly to a JVM method call. In fact, there's >> no significant difference between how Clojure compiles such code and how a >> Java compiler (such as javac) would compile such code. Clojure, too, >> explicitly handles boxing and unboxing of primitive types and other >> conveniences. Where Clojure slightly differs is in how it picks the correct >> method to call (it does method reflection, but only once during the >> compilation phase), and its coercion of types to match the called method -- >> after all, its type system is dynamic. >> >> [See clojure.lang.Compiler#InstanceMethodExpr, StaticMethodExpr, etc.] >> > > This just isn't true. If you don't provide any type hints, then the > constructor for InstanceMethodExpr sets method = null and warns you that > you're doing reflection (if you have the warn-on-reflection var set) > https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1364then > in the emit function, when method == null it compiles the invocation to > a call to call to Reflector.invokeInstanceMethod ( > https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1456and > > https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L25) > > which does reflection at runtime everytime you invoke the > method...*everytime*. With invokedynamic you can do the lookup in a > bootstrap method for the invokedynamic call site, you get a MethodHandle to > the method you found, and you cache it in a CallSite. Then the next time you > come through you just use the MethodHandle in the CallSite. That's a huge > performance boost. With invokedynamic you can do the same thing for static > field access. Clojure cannot optimize static field access like this. > > Also the Clojure Reflector class does a lot of mojo to try to find the > right method, and do boxing of arguments to match them up with the method > you want to invoke ( > https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L467and > > https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L427). > > If you call MethodHandle.invoke (versus MethodHandle.invokeExact) it does > that for you automatically, and presumably in the lower layers of the JVM > that Clojure cannot access, *and* the JVM knows more about what is going on > for optimization purposes. > > Clojure also does other things with arguments to functions, like collecting > and spreading rest args. These can be done with a chain of MethodHandles > which pushes the work down into the JVM. > > >> [snip] >> > Another note re: JRuby, is that it also has a very nice implementation of >> constants that makes use of invokedynamic, which ends up being somewhat >> similar to the problem of switching functions dynamically, but also uses a >> "SwitchPoint" mechanism to allow for very efficient concurrent mutability of >> constants. Indeed, JRuby makes use of every part of JSR-292! Clojure doesn't >> need consts, of course, because everything is immutable. >> > > Not everything in Clojure is immutable. The most basic element of Clojure's > dispatch (the Var) is mutable, and Clojure could benefit in the same way as > JRuby with SwitchPoints. Rich did work in 1.3 to basically keep track of > when the world of Vars changes, and when it does all of the cached Var > values get invalidated, and the next time a Var is used its value is > re-fetched. This is basically a SwitchPoint mechanism, and Sun nee Oracle > has already done the work for you, and other dynamic languages will be using > it, and everyone will benefit from improvements and optimizations except > Clojure, because we don't use invokedynamic. > > >> In summary, Clojure's dynamics are already handled as well as could be on >> the JVM, at least as far as I can see. >> > > For version 5 of the JVM, I agree, and if your goal is to maintain > compatibility with version 5 of the JVM, sure. However, there is lots of > invokedynamic goodness that Clojure could benefit from. Java 7 could be an > optional target that would tap into invokedynamic. You then get into the > debate about whether its going to be worth the effort, when what we have > already is about as good as you can do on Java 5. > > I'm not necessarily disagreeing with anyone who says it doesn't make sense > for Clojure to support invokedynamic given the current situation and desire > to maintain Java 5 compatibility. However, I do take issue with people > saying that Clojure cannot benefit from invokedynamic in principle. > > > Paul > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
