Hi Nicola, Congratulations - great work!
My main idea/feedback is that I'd love to see the "optimised" mode become able to emit code that matches Java / Scala performance in all cases. This would be a huge boost for people (like me) who do a lot of numerical / realtime work and would love to do this in pure Clojure rather than dropping down to Java all the time. Mostly, the difference in performance comes from using the correct static types, avoiding boxing, and avoiding all the dynamic checks/casting that Clojure currently does. 1) To make that work, we need to make more use of static typing / type inference to emit the right optimised bytecode for each situation. Sounds like that is in your plan already. 2) This in turn means we are going to have to "recompile" if we want to retain any sort of dynamic behaviour. The current system of independently mutable vars won't work (for obvious reasons: changing a var could easily violate the previous type inference assumptions) 3) Doing recompilation efficiently and incrementally implies keeping a dependency graph of compiled code., as well as retaining the source code in memory. 4) For sanity and correctness reasons, I think the best approach to this is having an "immutable environment", i.e. any action that redefines a var (e.g. "def") produces a new modified immutable environment, including updated dependencies. Fortunately persistent data structures make this cheap. 5) Immutable environments also offer a number of other advantages over the current mutable namespaces, e.g. better ability to handle circular references, the ability to store snapshots of past environments and the ability to fork environments to create new independent application instances. That's a rough sketch of what I think would work for a truly great Clojure 2.0 "optimised mode". If we do all this, then we will be in a very good place: a) We'll have the compiled performance equal to Java (or maybe even better, if we can exploit some JVM features that Java doesn't) b) We'll retain all the dynamic / interactive coding capabilities of Lisp, which is part of what makes Clojure great now. c) Immutable environments can evolve to replace the current mutable namespace system - which I think would be a big win in the long term, and is the right way to go philosophically if we take the "programming with values" stuff seriously. If you are interested and able to make even part of this happen in CinC, happy to discuss and contribute where I can! On Monday, 23 September 2013 04:59:30 UTC+8, Nicola Mometto wrote: > > > For the past 3 months I've been working as part of my GSoC project, > on a port of the clojure compiler and analyzer to clojure. > > Tomorrow the GSoC will end, so this is a report of what I've accomplished > with this project thus far; note that I'm not going to stop working on > CinC now that the GSoC is over. > > First, a link to the project if anybody is interested: > https://github.com/Bronsa/CinC > > What's there? > The project is made of 2 major components: the analyzer and the emitter. > > The analyzer is based heavly on the clojurescript analyzer, I've used > (successfully) the :children-keys approach to write a generic walker > over the AST and to implement multiple passes in order to > annotate the AST with all the information needed in order to compile to > JVM bytecode. > It should be noted that this analyzer provides all the information > collected by the clojure compiler (info on locals-clearing, loop-locals > invalidation etc) in a much more accessible way, exposing it all as > fields in a hash-map. > > The compiler takes this AST and constructs an AST representing the class > to be emitted, expressing the byte-code as a data-structure too, and it > subsequently interprets this AST evaluating the expression. > > The `doc` folder contains some further documentation on how the > analyzer/compiler works, it's not much yet but more is to come. > > What's not there yet? > While `cinc.compiler.jvm.bytecode/eval` is capable of evaluating all of > clojure special forms, primitive support mostly not in place, and should > be expected to be broken. > This means that some expression that need primitive support to work, for > example `defrecord` will not work. > > What's going to happen? > CinC is a project I wanted to work on for quite some time, and I'm not > going to abandon it now that the GSoC is over, I have things I want to > experiment with CinC (and I hope I'm not the only one), my current to-do > list is: > > * get primitive support fully working (including invokePrim) > * refactor :tag/:cast/:box handling > * standardize the AST format between CinC, clojurescript and > jvm.tools.analyzer, David Nolen and Ambrose B.S. are obviously who I'm > most looking forward to talking to about this, but any opinion will be > be greatly appreciated. > * have 2 compile target: one "normal" target that won't do any type > specialization and emit a dynamic bytecode (mostly for repl > experimentations), one "optimized" target that will do aggressive > tag inference and compile to more static code trying to avoid most of > the runtime reflection. > * experiment dynamically emitting invokePrim interfaces for primitive > types other than long or doubles > > > Ideas/feedbacks/contributions are greatly appreciated! > > Thanks, > Nicola > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
