Personally, I've recently fought machine-specific AOT breakage issues by forcing maven-clojure-plugin to only compile a single entry-point namespace, making compilation order deterministic again. Arguably, it should have worked in any order but this specific issue was hard to track down.
In any case, I think there is existing code that relies on the determinism. On Wed, Feb 10, 2016 at 11:36 AM Alex Miller <a...@puredanger.com> wrote: > Parallel loading is something I've looked at a little bit. Unfortunately > theory is easier in theory than in practice. :) While in many cases, it > probably works fine, it's possible for there to be problematic ambiguity > with which namespace is loaded first, especially for things that update > stateful things in the Clojure runtime (protocol extensions, multimethod > cases, namespaces, the dynamic classloader, etc). Perhaps you've heard that > mutable state and concurrency are problematic. :) I think it's been > interesting to see the issues that ClojureScript has found as they've > implemented parallel compilation as they are likely similar. > > Anyhow, it's certainly worth further consideration. > > > On Wednesday, February 10, 2016 at 4:53:06 AM UTC-6, Marc O'Morain wrote: >> >> Hi Alex, >> >> I've love to offer any help/test data on this that I can. At CircleCI we >> run (among other processes) a large Clojure app that takes 2 minutes to >> load on a top of line Macbook Pro, with Java 1.8 (Hotspot) . From my best >> efforts at profiling, this time is all spend in the Clojure Compiler and >> the JVM class loader, loading all transitive dependencies. Mike's analysis >> above is totally in line with what we see. >> >> One thing I have noticed is that all compilation runs in a single thread. >> In theory all namespaces required in an `ns` form can be loaded in parallel. >> >> If there is any profiling you would like me to run on the code-base, >> please let me know. >> >> Marc >> >> On 10 February 2016 at 08:22, Mikera <mike.r.anderson...@gmail.com> >> wrote: >> >>> Good initiative, I've filled in a response to the survey. >>> >>> One thing that strikes me is that the main issue with loading time is >>> the time required to transitively load and compile all the referred >>> namespaces (which can be a lot in a big project...). This in turn may >>> trigger class loading of Java libraries, IO as various resources are >>> initialised etc. An observation is that a lot of this is often not required >>> initially, so there *might* be a clever strategy to mitigate this through >>> laziness. >>> >>> This could be something like: >>> - When loading the current namespace, *don't* load referred namespaces >>> (yet) >>> - Create lazy placeholders within vars in the current namespace for (for >>> every def, defn etc.) >>> - Only when the placeholder is deref'd / invoked then compile the >>> relevant function and pull in dependencies. >>> - After first deref / invocation, replace the placeholder with the full >>> compiled function / value, so that subsequent access has no overhead >>> >>> This would be a pretty big change, and might break tools that make >>> assumptions about order of loading of namespaces... but I think it would >>> solve 90% of the boot time problems if implemented correctly. >>> >>> The other potentially big win would be concurrent loading of namespaces. >>> Guess you are looking at that already? >>> >>> >>> >>> On Wednesday, 10 February 2016 02:36:43 UTC+8, Alex Miller wrote: >>>> >>>> I'm doing some research on slow Clojure boot time and would be >>>> interested in collecting info about example use cases where it's been a >>>> problem for people. >>>> >>>> http://goo.gl/forms/eSpn8M5FNB >>>> >>>> I'm not expecting to release the results in any formal way, mostly >>>> looking to use it to pull out particular use cases and/or commonalities >>>> across anecdotes so that we can work on boot time problems that matter the >>>> most (and solutions likely to help the most). Any numbers you can provide >>>> would be great. >>>> >>>> Alex >>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clojure@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+unsubscr...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
