+1 for a common framework in maven-shared or similar. +0 for auto detecting changed scenarios. If someone changes a profile or the whole pom, then I'd expect he invokes a clean manually. We have to document this expectation of course.
LieGrue, strub ----- Original Message ----- > From: Anders Hammar <and...@hammar.net> > To: Maven Developers List <dev@maven.apache.org> > Cc: > Sent: Monday, August 27, 2012 9:57 AM > Subject: Re: improving incremental builds > >> I already started tweaking the m-compiler-plugin and found quite scary > issues. There is e.g. MCOMPILER-21 (open since 2005) which prevents > incremental > build and would kick in in your scenario. > > This is interesting. I've looked a bit at Mojo's JAXB plugin in the > context of detecting stale generated files (i.e. need to re-generate) > and it's similar to this. It would extremely nice if there would be a > common "framework" for handling incremental builds. > In addition to what's been mentioned in the jira (I just browsed it > quickly), we have cases when includes/excludes change, the sourceDir > changes, etc which should trigger cleanup and re-compilation. > > /Anders > >> >> I talked about 2 scenarios >> >> a.) all phases >= package, e.g. mvn verify, mvn install, ... Here we > have an artifact on the disc and could test for the md5 of them, right? >> >> b.) all phases < package. This is e.g. mvn compile or mvn test. In that > case we don't produce a jar/war/ear/... artifact but only get the files on > the disk linked in MavenProject#getProjectArtifacts()... file(). This is the > case where the maven-compiler-plugin kicks in. I'm currently in the process > of rewriting big parts of it, mainly I introduced >> b.1 a check for project.artifacts/project.testArtifacts .file() is a > local file path .isDirectory() and has files which are newer (actually >=) > than the buildStarted timestamp. >> b.2 check whether there are any *.java (sourceincludes) files which are > newer than their class files. >> >> In both cases I now force a FULL recompile of the module! Compiling only > parts produced classes which are actually broken! >> >> >> My approach is the following: compiling a bit too much is better than > missing some files which need compilation. Because the later is exactly the > reason why noone uses mvn compile but always do a mvn clean compile nowadays. > If > mvn compile is reliable, then we will end up being much faster than an > unconditional full compile on the whole project! >> >> >> LieGrue, >> strub >> >> >> PS: We need to move all our plugins which still use the > plugin-testing-harness to the maven-invoker-plugin as the test-harness is > broken > with sisu. (sisu changed the 'container' field from plexus original: > protected to 'private') >> >> PPS: how do we maintain the plexus-compiler-utils stuff? This contains some > weirdo bugs which should get fixed... >> >> >> >> ----- Original Message ----- >>> From: Olivier Lamy <ol...@apache.org> >>> To: Maven Developers List <dev@maven.apache.org>; Mark Struberg > <strub...@yahoo.de> >>> Cc: >>> Sent: Monday, August 27, 2012 9:13 AM >>> Subject: Re: improving incremental builds >>> >>> Hi, >>> Sounds good idea trying to improve that. >>> My question: on what is based the md5 calculation ? >>> If people don't use 'install' but only 'test' > (perso I use >>> that to >>> avoid too much io with jar creation etc..), so in such case we cannot >>> do md5 calculation on the produced artifact. >>> >>> 2012/8/26 Mark Struberg <strub...@yahoo.de>: >>>> Hi David! >>>> >>>>> So your idea is to find the list of changed modules and >>>>> then build that list with -amd? >>>> Yes, kind of. >>>> >>>> At the moment mvn -amd builds the dependents of the _current_ > module. If >>> you use my example and change BeanA.java, then run mvn -amd from the > root module >>> you will see that only moduleA gets rebuild and moduleB remains > original. >>> Because moduleB is not a dependent of the root module. >>>> >>>> But yes, I'm completely with you that we have most of the > ingredients >>> in the maven codebase already. At least for the start we could easily > detect >>> changed build results and add those to the 'amd' list. That > would >>> already be much better than what we have today imo. And this should be > pretty >>> easy to implement. >>>> >>>> Indeed, what I proposed goes a bit beyond -amd. I would not only > check the >>> current build tree like -amd does (even if that would be a good start) > but >>> remember the md5 of all the dependencies of each module (simply store > them in a >>> file in ./target/) And if you find a dependency which is not in the > list (e.g. >>> after upgrade from one version to another) or has a different md5 (this > covers >>> SNAPSHOT versions) , then we could force a full rebuild (mvn -amd) of > this >>> module. >>>> >>>> >>>> LieGrue, >>>> strub >>>> >>>> >>>> >>>> ----- Original Message ----- >>>>> From: David Jencks <david_jen...@yahoo.com> >>>>> To: Maven Developers List <dev@maven.apache.org> >>>>> Cc: >>>>> Sent: Sunday, August 26, 2012 8:34 AM >>>>> Subject: Re: improving incremental builds >>>>> >>>>> Is what you want different from what >>>>> >>>>> mvn -amd moduleA >>>>> >>>>> does? So your idea is to find the list of changed modules and > then >>> build that >>>>> list with -amd? >>>>> >>>>> thanks >>>>> david jencks >>>>> >>>>> On Aug 25, 2012, at 1:32 PM, Mark Struberg wrote: >>>>> >>>>>> Hi folks! >>>>>> >>>>>> After a short discussion with Kristian, I've created > a small >>> app with 2 >>>>> modules which shows a few problems with mavens incremental > build logic. >>>>>> And since incremental builds do not work well, people use >>>>>> >>>>>> $> mvn _clean_ install >>>>>> all the time. >>>>>> >>>>>> We could speed up the development effort heavily if we > make >>>>>> $> mvn install >>>>>> (without an upfront clean) more reliable. >>>>>> >>>>>> >>>>>> The sample [1] consists of moduleA and moduleB with BeanA > and >>> BeanB >>>>> respectively. >>>>>> BeanB has a private BeanA field and moduleB has a > dependency on >>> moduleA. >>>>>> >>>>>> If I change BeanA and just invoke mvn install then only > moduleA >>> gets >>>>> rebuilt. We currently do not rebuild moduleB, but we should do > to >>> create a >>>>> reliable output. >>>>>> >>>>>> In fact, the incremental build within a single module > already >>> works to some >>>>> degrees, but we must detect a change in dependencies and > trigger a full >>> rebuild >>>>> on all depending modules. This could be done by storing the > md5 of all >>>>> dependency artifacts and compare them on the next build. If > the md5 of >>> a >>>>> dependency did change, then we need to build the module full > cycle. >>>>>> Other ideas are welcome. Slaps as well if I forgot some > obvious >>> stuff and >>>>> all works well already. >>>>>> >>>>>> >>>>>> LieGrue, >>>>>> strub >>>>>> >>>>>> >>>>>> >>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>>> For additional commands, e-mail: > dev-h...@maven.apache.org >>>>>> >>>>> >>>>> >>>>> > --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>>> >>>> >>>> > --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>> For additional commands, e-mail: dev-h...@maven.apache.org >>>> >>> >>> >>> >>> -- >>> Olivier Lamy >>> Talend: http://coders.talend.com >>> http://twitter.com/olamy | http://linkedin.com/in/olamy >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >> For additional commands, e-mail: dev-h...@maven.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
