hmmm, interesting I think the issue is that I had to-do multiple pushes to nexus to get this to work (unfortunately) and perhaps maven central did not overwrite like it did with apache rather appending and meshing the poms together. The POM in maven central looks like a combination of what I was doing in https://issues.apache.org/jira/browse/KAFKA-974
Please create a JIRA however I suspect when we do another release (either 0.8.0-beta2 or 0.8.0) this will be correct and 1:1 from apache repo because I won't be doing multiple pushes like i had to-do here the weird parent block is required by apache when releasing http://www.apache.org/dev/publishing-maven-artifacts.html as I understand it and read the docs perhaps for 0.8.0-beta1 folks should just use the repo https://repository.apache.org/content/repositories/releases/ because matches the POM from the 0.8 branch great to be working all this stuff out in beta1 so we can have a nice clean 0.8 release =8^) On Mon, Jul 15, 2013 at 2:12 PM, Chris Riccomini <criccom...@linkedin.com>wrote: > Hey Guys, > > Digging into this more. Here's a fun fact: Maven Central's POM does not > match Apache release's. > > > http://search.maven.org/remotecontent?filepath=org/apache/kafka/kafka_2.9.2 > /0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom > > > https://repository.apache.org/content/groups/public/org/apache/kafka/kafka_ > 2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom > > Notice in Maven central that it has two <dependencies> blocks. That > doesn't exist in Apache release's. Weird. > > On a hunch, I explicitly added the Apache release repository as the first > repo that Gradle uses: > > allprojects { > repositories { > maven { > url 'https://repository.apache.org/content/groups/public' > } > mavenCentral() > } > } > > > This worked! I still have to manually add the exclusions: > > compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion") { > exclude module: 'jms' > exclude module: 'jmxtools' > exclude module: 'jmxri' > } > > > After that, I got a successful build. I still think the POM is pretty > broken, though. Here are the issues I see: > > 1. Maven central can't resolve it properly (POM is different from Apache > release). Have to use Apache release repo directly to get things to work. > 2. Exclusions must be manually applied even though they exist in Kafka's > POM already. I think Maven can handle this automatically, if the POM is > done right. > 3. Weird parent block in Kafka POMs that points to org.apache. > 4. Would be nice to publish kafka-test jars as well. > 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job. > > Shall I open Jiras for this stuff? > > Cheers, > Chris > > On 7/15/13 10:55 AM, "Chris Riccomini" <criccom...@linkedin.com> wrote: > > >Hey Guys, > > > >The problem persists, even when using the explicit URL Joe provided. > > > >I've also constructed a dummy Maven project, and tested Kafka dependencies > >there. That worked, but I had to explicit write the <exclusion> block. > > > >I might have to sick our local Gradle experts on this one... > > > >Cheers, > >Chris > > > >On 7/15/13 10:45 AM, "chetan conikee" <coni...@gmail.com> wrote: > > > >>I stumbled on a few issues when I was attempting to create a Scala based > >>Client (Producer/Consumer) project. > >> > >>Finally worked after revisions: > >> > >> > >>Here's the snippet of by build.sbt > >> > >>~~~~~~~~~~ > >> > >>/* scala versions and options */ > >>scalaVersion := "2.9.2" > >> > >> > >>libraryDependencies += > >> "org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1" excludeAll( > >> ExclusionRule(organization = "com.sun.jdmk"), > >> ExclusionRule(organization = "com.sun.jmx"), > >> ExclusionRule(organization = "javax.jms") > >> ) > >> > >> > >>/* you may need these repos */ > >>resolvers := Seq( > >> "maven" at "http://mvnrepository.com";, > >> "maven atrifactory" at "http://central.maven.org/maven2";, > >> "sonatype releases" at " > >>https://oss.sonatype.org/content/repositories/releases/";, > >> "sonatype snapshots" at " > >>https://oss.sonatype.org/content/repositories/snapshots/";, > >> "nexus releases" at " > >>https://oss.sonatype.org/service/local/staging/deploy/maven2";, > >> "Local Maven Repository" at "file://"+Path.userHome+"/.ivy2/repository" > >>) > >> > >>~~~~~~~~~~~~~~~~~~~~~~~~ > >> > >> > >>On Mon, Jul 15, 2013 at 10:39 AM, Chris Riccomini > >><criccom...@linkedin.com>wrote: > >> > >>> Hey Joe, > >>> > >>> The error I see is: > >>> > >>> 10:35:30.431 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build > >>> failed with an exception. > >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] > >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] * What went > >>>wrong: > >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] Could not > >>>resolve > >>> all dependencies for configuration ':samsa-kafka_2.9.2:compile'. > >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] > Could not > >>> resolve org.apache.kafka:kafka_2.9.2:0.8.0-beta1. > >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] Required by: > >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] > >>> samsa:samsa-kafka_2.9.2:0.7.0 > >>> 10:35:30.434 [ERROR] [org.gradle.BuildExceptionReporter] > null name > >>> not allowed > >>> > >>> All I could find on the subject was this: > >>> > >>> > >>> > http://forums.gradle.org/gradle/topics/illegalargumentexception_null_nam > >>>e > >>>_n > >>> > >>>ot_allowed_resolving_org_iso_relax_verifier_jaxp_validation_isorelax_jax > >>>p > >>>_b > >>> ridge_1_0-cavg1 > >>> > >>> > >>> As far as I can tell, SBT's "intransitive()" method disabled ALL > >>> transitive dependencies: > >>> > >>> "By default, these declarations fetch all project dependencies, > >>> transitively. In some instances, you may find that the dependencies > >>>listed > >>> for a project aren't necessary for it to build. Projects using the > >>>Felix > >>> OSGI framework, for instance, only explicitly require its main jar to > >>> compile and run. Avoid fetching artifact dependencies with either > >>> intransitive() or notTransitive(), as in this example:" > >>> > >>> > >>> -- https://github.com/harrah/xsbt/wiki/Library-Management > >>> > >>> > >>> I tried doing the Gradle equivalent of SBT's intransitive: > >>> > >>> compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion@jar") { > >>> transitive = false > >>> } > >>> > >>> The build still fails with that error. > >>> > >>> > >>> Cheers, > >>> > >>> Chris > >>> > >>> On 7/15/13 10:34 AM, "Joe Stein" <crypt...@gmail.com> wrote: > >>> > >>> >What is/are the error(s) you are getting? > >>> > > >>> >"intransitive" in the SBT line is so that SBT does not fetch the > >>>libraries > >>> >what is set for exclusion in the POM > >>> > > >>> >an example of an error if you don't do use what should be excluded is > >>>in > >>> >https://issues.apache.org/jira/browse/KAFKA-974 > >>> > > >>> > > >>> >On Mon, Jul 15, 2013 at 1:30 PM, Chris Riccomini > >>> ><criccom...@linkedin.com>wrote: > >>> > > >>> >> Hey Joe, > >>> >> > >>> >> First off, thanks for doing this! > >>> >> > >>> >> I'm trying to use the publication with Gradle, and I'm running into > >>> >> problems. It's giving me a funky exception. I did some digging, and > >>> >> apparently the exception implies that Gradle has been given a 1.0 > >>>pom. > >>> >> > >>> >> I took a look at the Kafka pom: > >>> >> > >>> >> > >>> >> > >>> >> > >>> > >>> > https://repository.apache.org/content/groups/public/org/apache/kafka/kaf > >>>k > >>> >>a_ > >>> >> 2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom > >>> >> > >>> >> > >>> >> It is indeed interesting. It appears to be a 2.0 pom, but it has a > >>> >>parent > >>> >> pointing to org.apache (?!). I dug around, and I haven't seen any > >>>other > >>> >> POMs with this style. I am not Maven expert, but at this point, it > >>>seems > >>> >> like the POMs Maven Central are somewhat broken in that regard. > >>> >> > >>> >> Have you successfully used the POMs with anything other than SBT? I > >>> >> noticed you're including "intransitive" in the SBT coordinates. Why > >>>is > >>> >> that? In general, we'd want Kafka's transitive dependencies to be > >>>pulled > >>> >> in, as well. Are you turning off transitivity because of an issue > >>> >>similar > >>> >> to the one I'm seeing? > >>> >> > >>> >> Thanks! > >>> >> Chris > >>> >> > >>> >> On 7/14/13 6:32 PM, "Joe Stein" <crypt...@gmail.com> wrote: > >>> >> > >>> >> >With SBT you can use 0.8.1-beta built with any of these four Scala > >>> >> >versions > >>> >> >in libraryDependencies now > >>> >> > > >>> >> >"org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1" intransitive() > >>> >> > > >>> >> >or > >>> >> > > >>> >> >"org.apache.kafka" % "kafka_2.9.1" % "0.8.0-beta1" intransitive() > >>> >> > > >>> >> >or > >>> >> > > >>> >> >"org.apache.kafka" % "kafka_2.8.2" % "0.8.0-beta1" intransitive() > >>> >> > > >>> >> >or > >>> >> > > >>> >> >"org.apache.kafka" % "kafka_2.8.0" % "0.8.0-beta1" intransitive() > >>> >> > > >>> >> > > >>> >> >/* > >>> >> >Joe Stein > >>> >> >http://www.linkedin.com/in/charmalloc > >>> >> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >>> >> >*/ > >>> >> > >>> >> > >>> > > >>> > > >>> >-- > >>> > > >>> >/* > >>> >Joe Stein > >>> >http://www.linkedin.com/in/charmalloc > >>> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >>> >*/ > >>> > >>> > > > > -- /* Joe Stein http://www.linkedin.com/in/charmalloc Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> */
