Hey Guys,
Digging into this more. Here's a fun fact: Maven Central's POM does not
match Apache release's.
http://search.maven.org/remotecontent?filepath=org/apache/kafka/kafka_2.9.2
/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
https://repository.apache.org/content/groups/public/org/apache/kafka/kafka_
2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
Notice in Maven central that it has two <dependencies> blocks. That
doesn't exist in Apache release's. Weird.
On a hunch, I explicitly added the Apache release repository as the first
repo that Gradle uses:
allprojects {
repositories {
maven {
url 'https://repository.apache.org/content/groups/public'
}
mavenCentral()
}
}
This worked! I still have to manually add the exclusions:
compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion") {
exclude module: 'jms'
exclude module: 'jmxtools'
exclude module: 'jmxri'
}
After that, I got a successful build. I still think the POM is pretty
broken, though. Here are the issues I see:
1. Maven central can't resolve it properly (POM is different from Apache
release). Have to use Apache release repo directly to get things to work.
2. Exclusions must be manually applied even though they exist in Kafka's
POM already. I think Maven can handle this automatically, if the POM is
done right.
3. Weird parent block in Kafka POMs that points to org.apache.
4. Would be nice to publish kafka-test jars as well.
5. Would be nice to have SNAPSHOT releases off of trunk using a Hudson job.
Shall I open Jiras for this stuff?
Cheers,
Chris
On 7/15/13 10:55 AM, "Chris Riccomini" <criccom...@linkedin.com> wrote:
>Hey Guys,
>
>The problem persists, even when using the explicit URL Joe provided.
>
>I've also constructed a dummy Maven project, and tested Kafka dependencies
>there. That worked, but I had to explicit write the <exclusion> block.
>
>I might have to sick our local Gradle experts on this one...
>
>Cheers,
>Chris
>
>On 7/15/13 10:45 AM, "chetan conikee" <coni...@gmail.com> wrote:
>
>>I stumbled on a few issues when I was attempting to create a Scala based
>>Client (Producer/Consumer) project.
>>
>>Finally worked after revisions:
>>
>>
>>Here's the snippet of by build.sbt
>>
>>~~~~~~~~~~
>>
>>/* scala versions and options */
>>scalaVersion := "2.9.2"
>>
>>
>>libraryDependencies +=
>> "org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1" excludeAll(
>> ExclusionRule(organization = "com.sun.jdmk"),
>> ExclusionRule(organization = "com.sun.jmx"),
>> ExclusionRule(organization = "javax.jms")
>> )
>>
>>
>>/* you may need these repos */
>>resolvers := Seq(
>> "maven" at "http://mvnrepository.com";,
>> "maven atrifactory" at "http://central.maven.org/maven2";,
>> "sonatype releases" at "
>>https://oss.sonatype.org/content/repositories/releases/";,
>> "sonatype snapshots" at "
>>https://oss.sonatype.org/content/repositories/snapshots/";,
>> "nexus releases" at "
>>https://oss.sonatype.org/service/local/staging/deploy/maven2";,
>> "Local Maven Repository" at "file://"+Path.userHome+"/.ivy2/repository"
>>)
>>
>>~~~~~~~~~~~~~~~~~~~~~~~~
>>
>>
>>On Mon, Jul 15, 2013 at 10:39 AM, Chris Riccomini
>><criccom...@linkedin.com>wrote:
>>
>>> Hey Joe,
>>>
>>> The error I see is:
>>>
>>> 10:35:30.431 [ERROR] [org.gradle.BuildExceptionReporter] FAILURE: Build
>>> failed with an exception.
>>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter]
>>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] * What went
>>>wrong:
>>> 10:35:30.432 [ERROR] [org.gradle.BuildExceptionReporter] Could not
>>>resolve
>>> all dependencies for configuration ':samsa-kafka_2.9.2:compile'.
>>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] > Could not
>>> resolve org.apache.kafka:kafka_2.9.2:0.8.0-beta1.
>>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter] Required by:
>>> 10:35:30.433 [ERROR] [org.gradle.BuildExceptionReporter]
>>> samsa:samsa-kafka_2.9.2:0.7.0
>>> 10:35:30.434 [ERROR] [org.gradle.BuildExceptionReporter] > null name
>>> not allowed
>>>
>>> All I could find on the subject was this:
>>>
>>>
>>>http://forums.gradle.org/gradle/topics/illegalargumentexception_null_nam
>>>e
>>>_n
>>>
>>>ot_allowed_resolving_org_iso_relax_verifier_jaxp_validation_isorelax_jax
>>>p
>>>_b
>>> ridge_1_0-cavg1
>>>
>>>
>>> As far as I can tell, SBT's "intransitive()" method disabled ALL
>>> transitive dependencies:
>>>
>>> "By default, these declarations fetch all project dependencies,
>>> transitively. In some instances, you may find that the dependencies
>>>listed
>>> for a project aren't necessary for it to build. Projects using the
>>>Felix
>>> OSGI framework, for instance, only explicitly require its main jar to
>>> compile and run. Avoid fetching artifact dependencies with either
>>> intransitive() or notTransitive(), as in this example:"
>>>
>>>
>>> -- https://github.com/harrah/xsbt/wiki/Library-Management
>>>
>>>
>>> I tried doing the Gradle equivalent of SBT's intransitive:
>>>
>>> compile("org.apache.kafka:kafka_$scalaVersion:$kafkaVersion@jar") {
>>> transitive = false
>>> }
>>>
>>> The build still fails with that error.
>>>
>>>
>>> Cheers,
>>>
>>> Chris
>>>
>>> On 7/15/13 10:34 AM, "Joe Stein" <crypt...@gmail.com> wrote:
>>>
>>> >What is/are the error(s) you are getting?
>>> >
>>> >"intransitive" in the SBT line is so that SBT does not fetch the
>>>libraries
>>> >what is set for exclusion in the POM
>>> >
>>> >an example of an error if you don't do use what should be excluded is
>>>in
>>> >https://issues.apache.org/jira/browse/KAFKA-974
>>> >
>>> >
>>> >On Mon, Jul 15, 2013 at 1:30 PM, Chris Riccomini
>>> ><criccom...@linkedin.com>wrote:
>>> >
>>> >> Hey Joe,
>>> >>
>>> >> First off, thanks for doing this!
>>> >>
>>> >> I'm trying to use the publication with Gradle, and I'm running into
>>> >> problems. It's giving me a funky exception. I did some digging, and
>>> >> apparently the exception implies that Gradle has been given a 1.0
>>>pom.
>>> >>
>>> >> I took a look at the Kafka pom:
>>> >>
>>> >>
>>> >>
>>> >>
>>>
>>>https://repository.apache.org/content/groups/public/org/apache/kafka/kaf
>>>k
>>> >>a_
>>> >> 2.9.2/0.8.0-beta1/kafka_2.9.2-0.8.0-beta1.pom
>>> >>
>>> >>
>>> >> It is indeed interesting. It appears to be a 2.0 pom, but it has a
>>> >>parent
>>> >> pointing to org.apache (?!). I dug around, and I haven't seen any
>>>other
>>> >> POMs with this style. I am not Maven expert, but at this point, it
>>>seems
>>> >> like the POMs Maven Central are somewhat broken in that regard.
>>> >>
>>> >> Have you successfully used the POMs with anything other than SBT? I
>>> >> noticed you're including "intransitive" in the SBT coordinates. Why
>>>is
>>> >> that? In general, we'd want Kafka's transitive dependencies to be
>>>pulled
>>> >> in, as well. Are you turning off transitivity because of an issue
>>> >>similar
>>> >> to the one I'm seeing?
>>> >>
>>> >> Thanks!
>>> >> Chris
>>> >>
>>> >> On 7/14/13 6:32 PM, "Joe Stein" <crypt...@gmail.com> wrote:
>>> >>
>>> >> >With SBT you can use 0.8.1-beta built with any of these four Scala
>>> >> >versions
>>> >> >in libraryDependencies now
>>> >> >
>>> >> >"org.apache.kafka" % "kafka_2.9.2" % "0.8.0-beta1" intransitive()
>>> >> >
>>> >> >or
>>> >> >
>>> >> >"org.apache.kafka" % "kafka_2.9.1" % "0.8.0-beta1" intransitive()
>>> >> >
>>> >> >or
>>> >> >
>>> >> >"org.apache.kafka" % "kafka_2.8.2" % "0.8.0-beta1" intransitive()
>>> >> >
>>> >> >or
>>> >> >
>>> >> >"org.apache.kafka" % "kafka_2.8.0" % "0.8.0-beta1" intransitive()
>>> >> >
>>> >> >
>>> >> >/*
>>> >> >Joe Stein
>>> >> >http://www.linkedin.com/in/charmalloc
>>> >> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> >> >*/
>>> >>
>>> >>
>>> >
>>> >
>>> >--
>>> >
>>> >/*
>>> >Joe Stein
>>> >http://www.linkedin.com/in/charmalloc
>>> >Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> >*/
>>>
>>>
>
