A 'mvn package -DskipTests -Pdist' (developer) with no changes takes 3.35mins
The biggest offenders are: [INFO] Apache Hadoop Common .............................. SUCCESS [43.299s] [INFO] Apache Hadoop HDFS ................................ SUCCESS [54.776s] [INFO] hadoop-yarn-api ................................... SUCCESS [15.466s] [INFO] hadoop-mapreduce .................................. SUCCESS [16.154s] [INFO] Apache Hadoop Distributed Copy .................... SUCCESS [9.657s] Everybody else is below 5secs A 'mvn test -DskipTests' with no changes 1.05mins (this should go down making the some codegen to be smart instead just compiling again) The biggest offenders are: [INFO] Apache Hadoop Common .............................. SUCCESS [9.121s] [INFO] Apache Hadoop HDFS ................................ SUCCESS [20.887s] [INFO] hadoop-yarn-api ................................... SUCCESS [6.664s] A few 2sec modules and the rest below 1sec Thanks. Alejandro On Thu, Feb 16, 2012 at 2:13 PM, Daryn Sharp <da...@yahoo-inc.com> wrote: > Wow, nice! How long is a rebuild with no changes? > > I'd be delighted if there was a dev profile that just did the bare minimum > to produce a working env. > > Daryn > > > On Feb 16, 2012, at 3:25 PM, Alejandro Abdelnur wrote: > > > I've just done some tweaks in the POMs and I'm cutting the 8.30min full > > dist build down to 3.45min. I'm removing all javadoc stuff from the dist > > profile. > > > > I think we should introduce a developer profile that does this. > > > > Also, the 'mvn install -DskipTests', to get the JARs in your m2 cache is > > taking 2mins. But we are building source JARs, the developer profile > could > > skip the generation of the source JARs, this would cut several seconds. > > > > Finally, note that to just build all JARs 'mvn clean test -DskipTests' it > > is taking 1.55mins. > > > > If folk agree I'll open a JIRA to optmize the build based on these > findings. > > > > Thxs. > > > > Alejandro > > > > On Wed, Feb 15, 2012 at 1:02 PM, Daryn Sharp <da...@yahoo-inc.com> > wrote: > > > >> Hi Sriram, > >> > >> You may want to try adding your source tree to Spotlight's privacy list, > >> otherwise it's busy trying to index files as fast as the build is > writing > >> them. If you are (un)fortunate enough to be running anti-virus sw, > then it > >> too is scurrying along behind your build checking every file. That's > >> enough to turn a normally silent MBP into a hair dryer. > >> > >> If you don't want to omit the entire project, because perhaps you want > to > >> use mdfind, then you can rename dirs to end with ".noindex". The > >> trick/hack is you can symlink the original dir name to the renamed > >> dir.noindex. > >> > >> But yes, I think the build is way to slow to be productive when used as > >> intended. The fact that people have to use custom tricks to build & > test > >> in a "reasonable" amount of time is a symptom of a problem... > >> > >> Daryn > >> > >> > >> On Feb 15, 2012, at 12:47 PM, Todd Lipcon wrote: > >> > >>> Hi Sriram, > >>> > >>> I also do -Dmaven.skip.javadoc=true usually and that cuts out a couple > >>> minutes. Unless you need a tarball, you can also drop -Ptar. > >>> > >>> When I'm just making quick changes confined to the java code, I > >>> usually just use "mvn -Dskiptests install" from within the hdfs > >>> project, then manually cp the resulting > >>> target/hadoop-hdfs-0.24.0-SNAPSHOT.jar into my install > >>> dir/share/hadoop/hdfs. Much faster than the full rebuild. > >>> > >>> -Todd > >>> > >>> On Wed, Feb 15, 2012 at 8:52 AM, Harsh J <ha...@cloudera.com> wrote: > >>>> Sriram, > >>>> > >>>> My package command didn't take as long with the same command -- where > >>>> do you see it getting stuck at most of the times? Which particular > >>>> target (right term?) seems to take the highest time to complete? > >>>> > >>>> On Wed, Feb 15, 2012 at 12:31 AM, Sriram Rao <srirams...@gmail.com> > >> wrote: > >>>>> Folks, > >>>>> > >>>>> The build takes forever: > >>>>> > >>>>> mvn package -Pdist -DskipTests -Dtar > >>>>> > >>>>> .... > >>>>> > >>>>> [INFO] > >>>>> > >> ------------------------------------------------------------------------ > >>>>> [INFO] BUILD SUCCESS > >>>>> [INFO] > >>>>> > >> ------------------------------------------------------------------------ > >>>>> [INFO] Total time: 6:25.399s > >>>>> [INFO] Finished at: Tue Feb 14 10:54:18 PST 2012 > >>>>> [INFO] Final Memory: 56M/123M > >>>>> [INFO] > >>>>> > >> ------------------------------------------------------------------------ > >>>>> > >>>>> This is on a mac with 8GB RAM. (The above case involved 0-lines of > >> code > >>>>> change; just making sure I had everything built!). > >>>>> > >>>>> Is there a faster way to get the thing to build? > >>>>> > >>>>> Sriram > >>>> > >>>> > >>>> > >>>> -- > >>>> Harsh J > >>>> Customer Ops. Engineer > >>>> Cloudera | http://tiny.cloudera.com/about > >>> > >>> > >>> > >>> -- > >>> Todd Lipcon > >>> Software Engineer, Cloudera > >> > >> > >
