On Thu, Aug 25, 2011 at 5:41 PM, Tal Liron <tal.li...@gmail.com> wrote:
> So, after setting up a JVM 7 environment to play with Clojure, and > enthusiastically rummaging through the codebase, I have good news and bad > news. :) > > So, when Clojure calls a function, it either already has the instance in > its entirety (a lambda) or it finds it by dereferencing a binding. Since all > functions are instances that implement IFn, the implementation can then call > invokeinterface, which is very efficient. > > [See clojure.lang.Compiler#InvokeExpr.emitArgsAndCall] > What I said in another post was that with invokedynamic (assuming you could just go whole hog into Java 7) you don't need the IFn interface. You just need a MethodHandle. So yes, I agree that given all the existing Clojure code, and the desire to stay compatible with Java 5 or 6 or whatever, invokedynamic isn't very compelling. If you were going to do a green field project, and didn't mind the Java 7 requirement, invokedynamic is very compelling, because you can basically push a bunch of code down into the JVM. This mean much less work, and it means you get to benefit from any improvements and optimizations that get built into the JVM. Even for Clojure having an optional Java 7 target may still be interesting in that it allows you to hook into all the work that will be done on invokedynamic to support the whole dynamic language ecosystem on the JVM. > Clojure can get away with this especially easily because the only variant > for function signatures in Clojure is arity. So, all we need is a simple > switch to call the IFn method with the correct arity. (Interestingly, this > means that Clojure has a fixed set of such signatures, and thus a fixed > upper limit to arity: 20, in case you were wondering.) > > In a language like Ruby, methods are not so free floating, and have much > more complex invocation styles as well as flexible signatures. (Clojure has > only one style: function calls are simple forms.) So, in order to use > invokeinterface, JRuby implementors would have had to create special classes > *per* method *per* invocation style if they wanted to be efficient. But this > is impossible, because so many classes would exhaust the perm-gen. For those > languages, invokedynamic is a terrific solution, because it lets them > dynamically link the implementation to the caller via a flexible > "bootstrapping" mechanism, allowing them to do entirely without the extra > class definition. Since all Clojure functions share essentially the same > class as well as interface, none of these challenges exist. > It's very possible that I'm missing something, but I'm not quite sure what you're getting at here. In JRuby, for instance, the method overloading is the same as in Clojure, it is based on arity. The slowdown is in dispatch where you have to check a singleton class, your class, your ancestors, then method_missing, etc. Once you've looked this all up you cache it to speed it up later. Then you want to invalidate the cache when the world changes. Clojure has similar needs. For instance, Clojure has protocol dispatch and multimethod dispatch as well as Var based dispatch. The multimethod dispatch does a look up in the derivation hierarchy. All of this stuff can change at runtime, just like in Ruby you can add methods to a class, change methods on a class, etc. I don't see how Clojure's dispatch needs are much different than what other dynamic languages are doing. > Another aspect is the immutability of these IFn instances: you can't > refactor them at runtime, all you can do is change the bindings to refer to > new functions. So, Clojure achieves runtime dynamics by letting you simply > rebind new functions to existing Vars, Refs, Atoms, etc., and the same > invocation route continues as usual. In a language like Ruby, the > bootstrapping mechanism of invokedynamic lets implementors change the base > linkage to reflect a new signature for the method. Again, a terrific JVM 7 > feature that Clojure simply does not need. > The "signature" of a JRuby method is not much different than a Clojure method. It is an arity overloaded method with generic arguments. They do similar things to lots of other dynamic languages (including Clojure) like generating a class with a bunch of different versions of the same method with different numbers of generic arguments ( http://jruby.org/git?p=jruby.git;a=blob;f=src/org/jruby/internal/runtime/methods/DynamicMethod.java;h=b65854c057c8a1acec57d6dad95158f08c960dcc;hb=HEAD#l203). invokedynamic isn't about method signatures it's about dynamic dispatch, which is what Clojure does. > Another issue I examined was how Clojure calls non-Clojure JVM code, > thinking that perhaps there invokedynamic would be useful. But, Clojure > again has a straightforward approach that poses no problems. Here, Clojure > very directly calls non-Clojure code using invokestatic, invokevirtual or > invokeinterface as appropriate. A Clojure form with the "." or ".." > notations translates quite directly to a JVM method call. In fact, there's > no significant difference between how Clojure compiles such code and how a > Java compiler (such as javac) would compile such code. Clojure, too, > explicitly handles boxing and unboxing of primitive types and other > conveniences. Where Clojure slightly differs is in how it picks the correct > method to call (it does method reflection, but only once during the > compilation phase), and its coercion of types to match the called method -- > after all, its type system is dynamic. > > [See clojure.lang.Compiler#InstanceMethodExpr, StaticMethodExpr, etc.] > This just isn't true. If you don't provide any type hints, then the constructor for InstanceMethodExpr sets method = null and warns you that you're doing reflection (if you have the warn-on-reflection var set) https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1364then in the emit function, when method == null it compiles the invocation to a call to call to Reflector.invokeInstanceMethod ( https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1456and https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L25) which does reflection at runtime everytime you invoke the method...*everytime*. With invokedynamic you can do the lookup in a bootstrap method for the invokedynamic call site, you get a MethodHandle to the method you found, and you cache it in a CallSite. Then the next time you come through you just use the MethodHandle in the CallSite. That's a huge performance boost. With invokedynamic you can do the same thing for static field access. Clojure cannot optimize static field access like this. Also the Clojure Reflector class does a lot of mojo to try to find the right method, and do boxing of arguments to match them up with the method you want to invoke ( https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L467and https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L427). If you call MethodHandle.invoke (versus MethodHandle.invokeExact) it does that for you automatically, and presumably in the lower layers of the JVM that Clojure cannot access, *and* the JVM knows more about what is going on for optimization purposes. Clojure also does other things with arguments to functions, like collecting and spreading rest args. These can be done with a chain of MethodHandles which pushes the work down into the JVM. > [snip] > Another note re: JRuby, is that it also has a very nice implementation of > constants that makes use of invokedynamic, which ends up being somewhat > similar to the problem of switching functions dynamically, but also uses a > "SwitchPoint" mechanism to allow for very efficient concurrent mutability of > constants. Indeed, JRuby makes use of every part of JSR-292! Clojure doesn't > need consts, of course, because everything is immutable. > Not everything in Clojure is immutable. The most basic element of Clojure's dispatch (the Var) is mutable, and Clojure could benefit in the same way as JRuby with SwitchPoints. Rich did work in 1.3 to basically keep track of when the world of Vars changes, and when it does all of the cached Var values get invalidated, and the next time a Var is used its value is re-fetched. This is basically a SwitchPoint mechanism, and Sun nee Oracle has already done the work for you, and other dynamic languages will be using it, and everyone will benefit from improvements and optimizations except Clojure, because we don't use invokedynamic. > In summary, Clojure's dynamics are already handled as well as could be on > the JVM, at least as far as I can see. > For version 5 of the JVM, I agree, and if your goal is to maintain compatibility with version 5 of the JVM, sure. However, there is lots of invokedynamic goodness that Clojure could benefit from. Java 7 could be an optional target that would tap into invokedynamic. You then get into the debate about whether its going to be worth the effort, when what we have already is about as good as you can do on Java 5. I'm not necessarily disagreeing with anyone who says it doesn't make sense for Clojure to support invokedynamic given the current situation and desire to maintain Java 5 compatibility. However, I do take issue with people saying that Clojure cannot benefit from invokedynamic in principle. Paul -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
