Please excuse the very naive question, but if the main problem seems to be compilation of loaded namespaces, what about aot? Does it help at all? If not, why? As far as I know, the JVM already does lazy loading with Java classes essentially in the way that Mike described.
On Wednesday, 10 February 2016, Marc O'Morain <m...@circleci.com> wrote: > Hi Alex, > > I've love to offer any help/test data on this that I can. At CircleCI we > run (among other processes) a large Clojure app that takes 2 minutes to > load on a top of line Macbook Pro, with Java 1.8 (Hotspot) . From my best > efforts at profiling, this time is all spend in the Clojure Compiler and > the JVM class loader, loading all transitive dependencies. Mike's analysis > above is totally in line with what we see. > > One thing I have noticed is that all compilation runs in a single thread. > In theory all namespaces required in an `ns` form can be loaded in parallel. > > If there is any profiling you would like me to run on the code-base, > please let me know. > > Marc > > On 10 February 2016 at 08:22, Mikera <mike.r.anderson...@gmail.com > <javascript:_e(%7B%7D,'cvml','mike.r.anderson...@gmail.com');>> wrote: > >> Good initiative, I've filled in a response to the survey. >> >> One thing that strikes me is that the main issue with loading time is the >> time required to transitively load and compile all the referred namespaces >> (which can be a lot in a big project...). This in turn may trigger class >> loading of Java libraries, IO as various resources are initialised etc. An >> observation is that a lot of this is often not required initially, so there >> *might* be a clever strategy to mitigate this through laziness. >> >> This could be something like: >> - When loading the current namespace, *don't* load referred namespaces >> (yet) >> - Create lazy placeholders within vars in the current namespace for (for >> every def, defn etc.) >> - Only when the placeholder is deref'd / invoked then compile the >> relevant function and pull in dependencies. >> - After first deref / invocation, replace the placeholder with the full >> compiled function / value, so that subsequent access has no overhead >> >> This would be a pretty big change, and might break tools that make >> assumptions about order of loading of namespaces... but I think it would >> solve 90% of the boot time problems if implemented correctly. >> >> The other potentially big win would be concurrent loading of namespaces. >> Guess you are looking at that already? >> >> >> >> On Wednesday, 10 February 2016 02:36:43 UTC+8, Alex Miller wrote: >>> >>> I'm doing some research on slow Clojure boot time and would be >>> interested in collecting info about example use cases where it's been a >>> problem for people. >>> >>> http://goo.gl/forms/eSpn8M5FNB >>> >>> I'm not expecting to release the results in any formal way, mostly >>> looking to use it to pull out particular use cases and/or commonalities >>> across anecdotes so that we can work on boot time problems that matter the >>> most (and solutions likely to help the most). Any numbers you can provide >>> would be great. >>> >>> Alex >>> >>> >>> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> <javascript:_e(%7B%7D,'cvml','clojure@googlegroups.com');> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> <javascript:_e(%7B%7D,'cvml','clojure%2bunsubscr...@googlegroups.com');> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com >> <javascript:_e(%7B%7D,'cvml','clojure%2bunsubscr...@googlegroups.com');>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > <javascript:_e(%7B%7D,'cvml','clojure@googlegroups.com');> > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > <javascript:_e(%7B%7D,'cvml','clojure%2bunsubscr...@googlegroups.com');> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com > <javascript:_e(%7B%7D,'cvml','clojure%2bunsubscr...@googlegroups.com');>. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
