Hey Joe,
Awesome. I've updated the Jira:
https://issues.apache.org/jira/browse/KAFKA-974
Cheers,
Chris
On 7/15/13 11:24 AM, "Joe Stein" <crypt...@gmail.com> wrote:
>hmmm, interesting
>
>I think the issue is that I had to-do multiple pushes to nexus to get this
>to work (unfortunately) and perhaps maven central did not overwrite like
>it
>did with apache rather appending and meshing the poms together. The POM
>in
>maven central looks like a combination of what I was doing in
>https://issues.apache.org/jira/browse/KAFKA-974
>
>Please create a JIRA however I suspect when we do another release (either
>0.8.0-beta2 or 0.8.0) this will be correct and 1:1 from apache repo
>because
>I won't be doing multiple pushes like i had to-do here
>
>the weird parent block is required by apache when releasing
>http://www.apache.org/dev/publishing-maven-artifacts.html as I understand
>it and read the docs
>
>perhaps for 0.8.0-beta1 folks should just use the repo
>https://repository.apache.org/content/repositories/releases/ because
>matches the POM from the 0.8 branch
>
>great to be working all this stuff out in beta1 so we can have a nice
>clean
>0.8 release =8^)
>
>
>
>On Mon, Jul 15, 2013 at 2:12 PM, Chris Riccomini
><criccom...@linkedin.com>wrote:
>
>> Hey Guys,
>>
>> Digging into this more. Here's a fun fact: Maven Central's POM does not
>> match Apache release's.
>>
>>
>>
>>http://search.maven.org/remotecontent?filepath=org/apache/kafka/kafka_2.9
>>.2
>> /0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
>>
>>
>>
>>https://repository.apache.org/content/groups/public/org/apache/kafka/kafk
>>a_
>> 2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
>>
>> Notice in Maven central that it has two <dependencies> blocks. That
>> doesn't exist in Apache release's. Weird.
>>
>> On a hunch, I explicitly added the Apache release repository as the
>>first
>> repo that Gradle uses:
>>
>> allprojects {
>> repositories {
>> maven {
>> url 'https://repository.apache.org/content/groups/public'
>> }
>> mavenCentral()
>> }
>> }
>>
>>
>> This worked! I still have to manually add the exclusions:
>>
>> compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion") {
>> exclude module: 'jms'
>> exclude module: 'jmxtools'
>> exclude module: 'jmxri'
>> }
>>
>>
>> After that, I got a successful build. I still think the POM is pretty
>> broken, though. Here are the issues I see:
>>
>> 1. Maven central can't resolve it properly (POM is different from Apache
>> release). Have to use Apache release repo directly to get things to
>>work.
>> 2. Exclusions must be manually applied even though they exist in Kafka's
>> POM already. I think Maven can handle this automatically, if the POM is
>> done right.
>> 3. Weird parent block in Kafka POMs that points to org.apache.
>> 4. Would be nice to publish kafka-test jars as well.
>> 5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson
>>job.
>>
>> Shall I open Jiras for this stuff?
>>
>> Cheers,
>> Chris
>>
>> On 7/15/13 10:55 AM, "Chris Riccomini" <criccom...@linkedin.com> wrote:
>>
>> >Hey Guys,
>> >
>> >The problem persists, even when using the explicit URL Joe provided.
>> >
>> >I've also constructed a dummy Maven project, and tested Kafka
>>dependencies
>> >there. That worked, but I had to explicit write the <exclusion> block.
>> >
>> >I might have to sick our local Gradle experts on this one...
>> >
>> >Cheers,
>> >Chris
>> >
>> >On 7/15/13 10:45 AM, "chetan conikee" <coni...@gmail.com> wrote:
>> >
>> >>I stumbled on a few issues when I was attempting to create a Scala
>>based
>> >>Client (Producer/Consumer) project.
>> >>
>> >>Finally worked after revisions:
>> >>
>> >>
>> >>Here's the snippet of by build.sbt
>> >>
>> >>~~~~~~~~~~
>> >>
>> >>/* scala versions and options */
>> >>scalaVersion := "2.9.2"
>> >>
>> >>
>> >>libraryDependencies +=
>> >> "org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1" excludeAll(
>> >> ExclusionRule(organization = "com.sun.jdmk"),
>> >> ExclusionRule(organization = "com.sun.jmx"),
>> >> ExclusionRule(organization = "javax.jms")
>> >> )
>> >>
>> >>
>> >>/* you may need these repos */
>> >>resolvers := Seq(
>> >> "maven" at "http://mvnrepository.com";,
>> >> "maven atrifactory" at "http://central.maven.org/maven2";,
>> >> "sonatype releases" at "
>> >>https://oss.sonatype.org/content/repositories/releases/";,
>> >> "sonatype snapshots" at "
>> >>https://oss.sonatype.org/content/repositories/snapshots/";,
>> >> "nexus releases" at "
>> >>https://oss.sonatype.org/service/local/staging/deploy/maven2";,
>> >> "Local Maven Repository" at
>>"file://"+Path.userHome+"/.ivy2/repository"
>> >>)
>> >>
>> >>~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >>
>> >>On Mon, Jul 15, 2013 at 10:39 AM, Chris Riccomini
>> >><criccom...@linkedin.com>wrote:
>> >>
>> >>> Hey Joe,
>> >>>
>> >>> The error I see is:
>> >>>
>> >>> 10:35:30.431 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE:
>>Build
>> >>> failed with an exception.
>> >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter]
>> >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] * What went
>> >>>wrong:
>> >>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] Could not
>> >>>resolve
>> >>> all dependencies for configuration ':samsa-kafka_2.9.2:compile'.
>> >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] > Could not
>> >>> resolve org.apache.kafka:kafka_2.9.2:0.8.0-beta1.
>> >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] Required
>>by:
>> >>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter]
>> >>> samsa:samsa-kafka_2.9.2:0.7.0
>> >>> 10:35:30.434 [ERROR] [org.gradle.BuildExceptionReporter] > null
>>name
>> >>> not allowed
>> >>>
>> >>> All I could find on the subject was this:
>> >>>
>> >>>
>> >>>
>> http://forums.gradle.org/gradle/topics/illegalargumentexception_null_nam
>> >>>e
>> >>>_n
>> >>>
>>
>>>>>ot_allowed_resolving_org_iso_relax_verifier_jaxp_validation_isorelax_j
>>>>>ax
>> >>>p
>> >>>_b
>> >>> ridge_1_0-cavg1
>> >>>
>> >>>
>> >>> As far as I can tell, SBT's "intransitive()" method disabled ALL
>> >>> transitive dependencies:
>> >>>
>> >>> "By default, these declarations fetch all project dependencies,
>> >>> transitively. In some instances, you may find that the dependencies
>> >>>listed
>> >>> for a project aren't necessary for it to build. Projects using the
>> >>>Felix
>> >>> OSGI framework, for instance, only explicitly require its main jar
>>to
>> >>> compile and run. Avoid fetching artifact dependencies with either
>> >>> intransitive() or notTransitive(), as in this example:"
>> >>>
>> >>>
>> >>> -- https://github.com/harrah/xsbt/wiki/Library-Management
>> >>>
>> >>>
>> >>> I tried doing the Gradle equivalent of SBT's intransitive:
>> >>>
>> >>> compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion@jar") {
>> >>> transitive = false
>> >>> }
>> >>>
>> >>> The build still fails with that error.
>> >>>
>> >>>
>> >>> Cheers,
>> >>>
>> >>> Chris
>> >>>
>> >>> On 7/15/13 10:34 AM, "Joe Stein" <crypt...@gmail.com> wrote:
>> >>>
>> >>> >What is/are the error(s) you are getting?
>> >>> >
>> >>> >"intransitive" in the SBT line is so that SBT does not fetch the
>> >>>libraries
>> >>> >what is set for exclusion in the POM
>> >>> >
>> >>> >an example of an error if you don't do use what should be excluded
>>is
>> >>>in
>> >>> >https://issues.apache.org/jira/browse/KAFKA-974
>> >>> >
>> >>> >
>> >>> >On Mon, Jul 15, 2013 at 1:30 PM, Chris Riccomini
>> >>> ><criccom...@linkedin.com>wrote:
>> >>> >
>> >>> >> Hey Joe,
>> >>> >>
>> >>> >> First off, thanks for doing this!
>> >>> >>
>> >>> >> I'm trying to use the publication with Gradle, and I'm running
>>into
>> >>> >> problems. It's giving me a funky exception. I did some digging,
>>and
>> >>> >> apparently the exception implies that Gradle has been given a 1.0
>> >>>pom.
>> >>> >>
>> >>> >> I took a look at the Kafka pom:
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>>
>> >>>
>> https://repository.apache.org/content/groups/public/org/apache/kafka/kaf
>> >>>k
>> >>> >>a_
>> >>> >> 2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
>> >>> >>
>> >>> >>
>> >>> >> It is indeed interesting. It appears to be a 2.0 pom, but it has
>>a
>> >>> >>parent
>> >>> >> pointing to org.apache (?!). I dug around, and I haven't seen any
>> >>>other
>> >>> >> POMs with this style. I am not Maven expert, but at this point,
>>it
>> >>>seems
>> >>> >> like the POMs Maven Central are somewhat broken in that regard.
>> >>> >>
>> >>> >> Have you successfully used the POMs with anything other than
>>SBT? I
>> >>> >> noticed you're including "intransitive" in the SBT coordinates.
>>Why
>> >>>is
>> >>> >> that? In general, we'd want Kafka's transitive dependencies to be
>> >>>pulled
>> >>> >> in, as well. Are you turning off transitivity because of an issue
>> >>> >>similar
>> >>> >> to the one I'm seeing?
>> >>> >>
>> >>> >> Thanks!
>> >>> >> Chris
>> >>> >>
>> >>> >> On 7/14/13 6:32 PM, "Joe Stein" <crypt...@gmail.com> wrote:
>> >>> >>
>> >>> >> >With SBT you can use 0.8.1-beta built with any of these four
>>Scala
>> >>> >> >versions
>> >>> >> >in libraryDependencies now
>> >>> >> >
>> >>> >> >"org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1"
>>intransitive()
>> >>> >> >
>> >>> >> >or
>> >>> >> >
>> >>> >> >"org.apache.kafka" % "kafka_2.9.1" % "0.8.0-beta1"
>>intransitive()
>> >>> >> >
>> >>> >> >or
>> >>> >> >
>> >>> >> >"org.apache.kafka" % "kafka_2.8.2" % "0.8.0-beta1"
>>intransitive()
>> >>> >> >
>> >>> >> >or
>> >>> >> >
>> >>> >> >"org.apache.kafka" % "kafka_2.8.0" % "0.8.0-beta1"
>>intransitive()
>> >>> >> >
>> >>> >> >
>> >>> >> >/*
>> >>> >> >Joe Stein
>> >>> >> >http://www.linkedin.com/in/charmalloc
>> >>> >> >Twitter: @allthingshadoop
>><http://www.twitter.com/allthingshadoop>
>> >>> >> >*/
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> >--
>> >>> >
>> >>> >/*
>> >>> >Joe Stein
>> >>> >http://www.linkedin.com/in/charmalloc
>> >>> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> >>> >*/
>> >>>
>> >>>
>> >
>>
>>
>
>
>--
>
>/*
>Joe Stein
>http://www.linkedin.com/in/charmalloc
>Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>*/
