Paul, I'm going to combine two posts of yours in to one, if I may, and answer some of your points.
(Aaron, can we add some of this to the wiki, too? I'm including at least one correction) *-- For version 5 of the JVM, I agree, and if your goal is to maintain compatibility with version 5 of the JVM, sure. However, there is lots of invokedynamic goodness that Clojure could benefit from.* I can think of a few ways in which it would be possible to distribute a clojure.jar that supports JVM 7 features while still falling back to JVM 5 compatibility. So, I don't think this was ever a barrier. But you obviously unconvinced by my report! Thank you for challenging me on this, As I said, I'm by no means 100% sure I exhausted the issue. I would encourage you to try out your ideas. *-- If you don't provide any type hints, then the constructor for InstanceMethodExpr sets method = null and warns you that you're doing reflection (if you have the warn-on-reflection var set) https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1364 then in the emit function, when method == null it compiles the invocation to a call to call to Reflector.invokeInstanceMethod (https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L1456 and https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Reflector.java#L25) which does reflection at runtime everytime you invoke the method...*everytime*. With invokedynamic you can do the lookup in a bootstrap method for the invokedynamic call site, you get a MethodHandle to the method you found, and you cache it in a CallSite. Then the next time you come through you just use the MethodHandle in the CallSite. That's a huge performance boost. With invokedynamic you can do the same thing for static field access. Clojure cannot optimize static field access like this. * You are right! This is perhaps the only place where invokedynamic can be useful for Clojure. *However*, to me this is not a big deal, because the workaround of allowing for type hinting solves the problem just as well, and perhaps even more efficiently from the compiler's point of view. All it does is require some extra annotations by the programmer. The fact that Clojure emits a warning seems good enough for me, personally. *-- I may be missing something, but I don't see how JRuby needs different classes.* I recommend this old writeup by Charles Nutter for a good explanation of why JRuby *could* have used classes to solve the optimization problem (JSR-292 changed a lot since 2008, but the principles remain pretty much the same): http://blog.headius.com/2008/09/first-taste-of-invokedynamic.html I would add one emphasis to his explanation -- He point outs, very rightly, that the JVM has no real static typing at the low level, *except* when it comes to linking methods. But there is in fact another serious limitation of the JVM that comes into play here, which might seem very obvious to him but not so much to Clojure folk: the smallest container unit of code for the JVM is a class. There is no such thing as a free-floating procedure that you can just invoke whenever. (Changing this would be a revolution in the JVM, because so much depends on the ClassLoader mechanism, from security to modularity in OSGi.) This means that any time you want to reuse code you need to create a class. (I forget which verson if the JVM it was -- 5? -- that it become possible for the garbage collector to unload classes. This was a great feature for dynamic languages like Rhino, which generated a lot of classes during runtime to solve different problems in dynamics.) The class problem even worse for dynamic languages, because classes cannot be changed runtime, so if you hoped that you could just create one huge class and keep adding static methods to as your "free-floating" code, this again. This is a problem not only for JRuby, but also for implementing delegates in the Java, a long-request feature. Java thus needs to do ridiculous things like this, which in C# (for example) would be done so much more elegantly: new Thread(new Runnable() { public void run() { ThisIsMyFakeDelegate(); }).start(); What we did above was create a new (anonymous) class as well as an instance! Clojure does essentially the same thing per fn, extending AFn instead of implementing Runnable. * Correction: In my original report, I said that Clojure creates only instances of AFn, not classes. I was misstating the situation: it's exactly one class and one instance per fn. Clojure extends AFn and simply overrides the correct method according to arity. (Perhaps I should emphasize that all method arguments are also of type Object, since no type checking is done explicitly or even known at compile time. So, no further method signatures are necessary other than one per number of Object arguments.) There is no problem with proliferation of classes here, because Clojure does not normally create many new functions at runtime. (I would only see this happening if you somehow embedded the REPL into Clojure and evaled code that created functions. But, unless you bind them somewhere, the classes would be garbage collected. If you are binding them, you should reconsider your solution, as something like that would probably cause any Lisp to choke...) *-- Clojure has similar needs. For instance, Clojure has protocol dispatch and multimethod dispatch as well as Var based dispatch. The multimethod dispatch does a look up in the derivation hierarchy. All of this stuff can change at runtime, just like in Ruby you can add methods to a class, change methods on a class, etc. I don't see how Clojure's dispatch needs are much different than what other dynamic languages are doing. * So, perhaps I'm not understanding how you are using the term "change at runtime" to describe both JRuby regular dispatch and Clojure's multimethod. Actually, this is a good place to again explain why Clojure is so different, and why I didn't even consider (perhaps prematurely?) multimethod as a place where invokedynamic could help: Ruby lets you change classes and modules at runtime, because from Ruby's perspective a class (or module) is simply a dictionary of methods. So, in such cases it's perfect that the CallSite can be invalidated, and force the JVM to call your bootstrapping method again every time you change the dict. So, "dynamic" in this sense means that the class/module dict is mutable, and invokedynamic allows for an elegant optimization. In Clojure, the "class/module" construct does not exist, and functions are bound to Vars/Refs/Atoms, if they are bound at all, which must always be dereferenced anyway. (On further thought, I'm wondering now if we can do something with invokedynamic here after all. We may be handle such bound functions through invokedynamic linkage instead via dereferencing, effectively moving the binding to a deeper level. This may involve very different implementations per Var/Ref/Atom respectively. I'm going to look into this some more -- we have to be sure not to break Clojure's semantics and limit its flexibility here.) Clojure's multimethod is "dynamic" in an entirely different way: the correct function is selected at runtime *according to the arguments* of the function. JVM-level linkage to the selected method would make no sense, because you'd be calling your bootstrapping method every single call! You can argue that Clojure multimethods are inherently inefficient, and perhaps they are. One solution I could see is we allow to expand Clojure's deftypes into something more like classes/modules in other languages, in that deftypes could encapsulate functions as well as fields. But, this would turn Clojure into essentially object-oriented language, and I sincerely doubt Rich would allow for that to happen. :) Hope this helps clarify things! I don't think I've answered every single one of your points, but this discussion is already probably too long for more readers. Again, I encourage you to try to implement your ideas. The more eyes are on this problem, the better. -Tal -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
