Hey Josh, I spent some time playing with the sbt-pom-reader and actually I think it might suite our needs pretty well. Basically our situation is that Maven has more mature support for packaging (shading, official assembly plug-in) and is used by pretty much every downstream packager of Spark. SBT is used by the developers heavily for incremental compilation, the console, etc. So far we've been maintaining entirely isolated builds in sbt and maven which is a huge development cost and something that's lead to divergence over time.
The sbt-pom-reader I think could give us the best of both worlds. But it doesn't support profiles in Maven which we use pretty heavily. I played a bit this week and tried to get it to work without much luck: https://github.com/pwendell/sbt-pom-reader/commit/ca1f1f2d6bf8891acb7212facf4807baaca8974d Any pointers or interest in helping us with that feature? I think with that in place we might have a good solution where the dependencies are declared in one place (poms) and we use Maven for packaging but we can use sbt for day to day development. On Fri, Mar 14, 2014 at 9:41 AM, Evan Chan <e...@ooyala.com> wrote: > Jsuereth - > > Thanks for jumping on this thread. There are a couple things which > would help SBT IMHO (w.r.t. issues mentioned here): > > - A way of generating a project-wide pom.xml (I believe make-pom only > generates it for each sub-project) > - and probably better or more feature-complete POMs, but the Maven > folks here can speak to that > - Make the sbt-pom-reader plugin officially part of SBT, and make it > work well (again the Maven folks need to jump in here, though plugins > can't be directly translated) > - Have the sbt-assembly plugin officially maintained by Typesafe, or > part of SBT.... most Maven folks expect not to have to include a > plugin to generate a fat jar, and it's a pretty essential plugin for > just about every SBT project > - Also there is no equivalent (AFAIK) to the maven shader plugin. > > I also wish that the dependency-graph plugin was included by default, > but that's just me :) > > -Evan > > > On Fri, Mar 14, 2014 at 6:47 AM, jsuereth <joshua.suer...@gmail.com> wrote: >> Hey guys - >> >> If there's anything we can do to improve the sbt experience, let me know. >> I'd be extremely interested to see how/where there are issues integrating >> sbt with the existing Hadoop ecosystem. >> >> Particularly the difficulties in using Sbt + Maven together (something which >> tends block more than just spark from adopting sbt). >> >> I'm more than happy to listen and see what we can do on the sbt side to make >> this as seamless as possible for all parties. >> >> Thanks! >> >> >> >> -- >> View this message in context: >> http://apache-spark-developers-list.1001551.n3.nabble.com/DISCUSS-Necessity-of-Maven-and-SBT-Build-in-Spark-tp2315p5682.html >> Sent from the Apache Spark Developers List mailing list archive at >> Nabble.com. > > > > -- > -- > Evan Chan > Staff Engineer > e...@ooyala.com |
