Re: [SPARK-3324] make yarn module as a unified maven jar project

2014-08-31 Thread Sean Owen
Yes, alpha and stable need to stay in two separate modules. I think this is a little less standard than simply having three modules: common, stable, alpha. On Sun, Aug 31, 2014 at 1:32 PM, Yi Tian wrote: > Hi Sean > > Before compile-time, maven could dynamically add either stable or alpha > sour

Re: [SPARK-3324] make yarn module as a unified maven jar project

2014-08-31 Thread Yi Tian
Hi Sean Before compile-time, maven could dynamically add either stable or alpha source to the yarn/ project. So there are no incompatible at the compile-time. Here are an example: yarn/pom.xml org.codehaus.mojo build-helper-maven-plugin a

Re: [SPARK-3324] make yarn module as a unified maven jar project

2014-08-31 Thread Sean Owen
This isn't possible since the two versions of YARN are mutually incompatible at compile-time. However see my comments about how this could be restructured to be a little more standard, and so that IntelliJ would parse it out of the box. Still I imagine it is not worth it if YARN alpha will go away

[SPARK-3324] make yarn module as a unified maven jar project

2014-08-30 Thread Yi Tian
Hi everyone! I found the YARN module has nonstandard path structure like: ${SPARK_HOME} |--yarn |--alpha (contains yarn api support for 0.23 and 2.0.x) |--stable (contains yarn api support for 2.2 and later) | |--pom.xml (spark-yarn) |--common (Common codes not depending