[
https://issues.apache.org/jira/browse/KUDU-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877502#comment-17877502
]
Joe McDonnell commented on KUDU-3610:
-------------------------------------
After taking another look, there is another dimension to this. It also matters
whether the dependency has the "compile" scope vs the "runtime" scope. In the
old kudu-client.pom, slf4j is marked as a compile dependency, but in the new
one it is marked as runtime. This same is true for stumbleupon async. I think
that indicates that compileUnshaded is not working as expected.
I'm also starting to think that this code is not working:
[https://github.com/apache/kudu/blob/e456bc775805d4555d99ce12d04e3ca0b8950760/java/gradle/shadow.gradle#L126-L161]
When I run gradlew with the "-i" option to turn on info logging, I expect this
log line to get hit:
[https://github.com/apache/kudu/blob/e456bc775805d4555d99ce12d04e3ca0b8950760/java/gradle/shadow.gradle#L148]
I see this log line on older Kudu versions, but I don't see it after the gradle
upgrade.
Here's a table of old versus new to try to make this clearer:
||Dependency||Before gradle upgrade||After gradle upgrade||
|com.stumbleupon:async|compile scope|runtime scope|
|org.slf4j:slf4j-api|compile scope|runtime scope|
|io.netty:netty-all|not present|runtime scope|
|org.apache.kudu:kudu-proto|not present|runtime scope|
|com.google.protobuf:protobuf-java|not present|runtime scope|
|com.google.guava:guava|not present|runtime scope|
|io.micrometer:micrometer-core|not present|runtime scope|
|com.sangupta:murmur|not present|runtime scope|
|com.google.code.findbugs:jsr305|compile optional|not present|
|org.apache.yetus:audience-annotations|compile optional|not present|
|junit:junit|test scope|not present|
|org.apache.kudu:kudu-test-utils|test scope|not present|
|org.apache.logging.log4j:log4j-api|test scope|not present|
|org.apache.logging.log4j:log4j-core|test scope|not present|
|org.apache.logging.log4j:log4j-slf4j-impl|test scope|not present|
|org.mockito:mockito-core|test scope|not present|
The piece of this that Impala cares about are the compile vs runtime
differences and the not present vs runtime differences. I don't think Impala
cares as much about the compile optional or test scope differences.
> KUDU-3551 introduced new unnecessary dependencies for kudu-client.jar
> ---------------------------------------------------------------------
>
> Key: KUDU-3610
> URL: https://issues.apache.org/jira/browse/KUDU-3610
> Project: Kudu
> Issue Type: Bug
> Components: java
> Affects Versions: 1.18.0
> Reporter: Joe McDonnell
> Assignee: Zoltan Chovan
> Priority: Major
> Attachments: kudu-client-a9d42cdb84.pom, kudu-client-e742f86f6d.pom
>
>
> When working on fixing IMPALA-13309, I was upgrading Kudu from
> [e742f86f6d|https://github.com/apache/kudu/commit/e742f86f6d] to
> [a9d42cdb84|https://github.com/apache/kudu/commit/a9d42cdb8438858cf1bd18c203d1687ed5734346],
> which is past the upgrade to Gradle 7.6.4 in
> [e456bc7758|https://github.com/apache/kudu/commit/e456bc775805d4555d99ce12d04e3ca0b8950760]
> (KUDU-3551). When using this new version, Impala's build fails with a
> message like this:
>
> {noformat}
> 15:24:49 [ERROR] Failed to execute goal on project impala-frontend: Could not
> resolve dependencies for project
> org.apache.impala:impala-frontend:jar:4.5.0-SNAPSHOT: Failed to collect
> dependencies at org.apache.kudu:kudu-client:jar:a9d42cdb84 ->
> org.apache.kudu:kudu-proto:jar:a9d42cdb84: Failed to read artifact descriptor
> for org.apache.kudu:kudu-proto:jar:a9d42cdb84: Could not transfer artifact
> org.apache.kudu:kudu-proto:pom:a9d42cdb84 from/to impala.cdp.repo (yadda
> yadda yadda)
> {noformat}
>
> Impala builds and publishes kudu-client.jar, but we currently don't publish
> kudu-proto.jar. This was fine on the old Kudu version, because
> kudu-client.jar is a shaded jar and does not have a runtime dependency on
> kudu-proto.jar. After the upgrade to Gradle 7.6.4, kudu-client.jar gains a
> runtime dependency on kudu-proto.jar and a few other dependencies (guava,
> netty, etc). These extra runtime dependencies should not be needed, because
> they are included in the shaded jar.
> I have attached the kudu-client.pom files from before and after for
> comparison:
> Before: [^kudu-client-e742f86f6d.pom]
> After: [^kudu-client-a9d42cdb84.pom]
> To produce this file during development, run this from the java directory:
> {noformat}
> ./gradlew -Dmaven.repo.local=/path/to/a/local/directory
> :kudu-client:publishToMavenLocal{noformat}
> Then, the new pom file will be under the
> /path/to/a/local/directory/org/apache/kudu/kudu-client/1.18.0-SNAPSHOT/
> directory.
> It would be good for the dependencies to be as restricted as before. I think
> we can get around some of these dependencies by using "compileOnly" rather
> than "implementation".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
