nickwallen commented on a change in pull request #1523: METRON-2232 Upgrade to
Hadoop 3.1.1
URL: https://github.com/apache/metron/pull/1523#discussion_r338651799
##########
File path: metron-platform/metron-hbase-server/pom.xml
##########
@@ -57,16 +63,34 @@
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <!-- We depend on this for some Zookeeper upload operations -->
+ <groupId>org.apache.metron</groupId>
+ <artifactId>stellar-common</artifactId>
+ <version>${project.parent.version}</version>
+ <classifier>uber</classifier>
+ </dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-enrichment-common</artifactId>
<version>${project.parent.version}</version>
+ <exclusions>
+ <exclusion>
Review comment:
I've been suspicious of using wildcard exclusions because of differences
between what Maven reports it is doing versus what it actually does when using
these. I dug into this a little more and here is what I've found.
(1) Wildcard exclusions do indeed work when building the jar. By
experimentation I can see that the dependencies from `metron-enrichment-common`
are not being included in the uber jar, as we'd expect.
(2) The exclusion is reported correctly when using `mvn dependency:tree`.
The output tells me below that no additional transitives are pulled-in.
```
~/tmp/metron-pr1523/metron-platform/metron-hbase-server
$ mvn dependency:tree
...
11:40:35,195 [INFO] +-
org.apache.metron:metron-enrichment-common:jar:0.7.2:compile
11:40:35,195 [INFO] +-
org.apache.metron:metron-hbase-common:jar:0.7.2:compile
11:40:35,195 [INFO] +- org.apache.hbase:hbase-common:jar:2.0.2:provided
...
```
(3) Oddly, when using `mvn dependency:tree -Dverbose` it makes it seem that
the dependencies are NOT excluded. This output makes it seem that it is
pulling in `hbase-client` as a transitive when it is not.
```
~/tmp/metron-pr1523/metron-platform/metron-hbase-server
$ mvn dependency:tree -Dverbose
...
11:38:37,776 [INFO] +-
org.apache.metron:metron-enrichment-common:jar:0.7.2:compile
11:38:37,776 [INFO] | +-
(com.github.ben-manes.caffeine:caffeine:jar:2.6.2:compile - omitted for
duplicate)
11:38:37,776 [INFO] | +- org.apache.hbase:hbase-client:jar:2.0.2:compile
11:38:37,776 [INFO] | | +-
org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:2.1.0:compile
11:38:37,776 [INFO] | | +-
(org.apache.hbase:hbase-common:jar:2.0.2:compile - omitted for duplicate)
11:38:37,776 [INFO] | | +-
org.apache.hbase:hbase-hadoop-compat:jar:2.0.2:compile
11:38:37,776 [INFO] | | | +-
(org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.1.0:compile -
omitted for duplicate)
11:38:37,776 [INFO] | | | +- (org.slf4j:slf4j-api:jar:1.7.25:compile -
omitted for duplicate)
11:38:37,776 [INFO] | | | +-
org.apache.hbase:hbase-metrics-api:jar:2.0.2:compile
11:38:37,776 [INFO] | | | | +-
(org.apache.hbase:hbase-common:jar:2.0.2:compile - omitted for duplicate)
11:38:37,776 [INFO] | | | | +- (org.slf4j:slf4j-api:jar:1.7.25:compile
- omitted for duplicate)
11:38:37,777 [INFO] | | | | +-
(org.apache.commons:commons-lang3:jar:3.6:compile - omitted for conflict with
3.2)
11:38:37,777 [INFO] | | | | +-
(org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.1.0:compile -
omitted for duplicate)
11:38:37,777 [INFO] | | | | \-
(org.apache.yetus:audience-annotations:jar:0.5.0:compile - omitted for
duplicate)
11:38:37,777 [INFO] | | | \-
(org.apache.yetus:audience-annotations:jar:0.5.0:compile - omitted for
duplicate)
11:38:37,777 [INFO] | | +-
org.apache.hbase:hbase-hadoop2-compat:jar:2.0.2:compile
```
I am not sure if this is a bug or is intentional when using `-Dverbose`, but
it is something we will have to keep in mind when investigating classpath
issues when wildcard exclusions are in-play.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
