[
https://issues.apache.org/jira/browse/KUDU-3157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17196498#comment-17196498
]
ASF subversion and git services commented on KUDU-3157:
-------------------------------------------------------
Commit 6953f53fb1f0ef617f596ec89690ab0e1adfc4e4 in kudu's branch
refs/heads/master from Grant Henke
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=6953f53 ]
KUDU-3157: Ensure we don’t shade slf4j classes
We should almost never shade slf4j classes but due to a behavior
issue in the Shadow Gradle plugin, exclusions from parent
modules are not respected in modules that use them. This meant
that we were shading slf4j classes into the kudu-spark module
jars and any modules that depend on kudu-spark.
This patch fixes the issue by universally excluding slf4j from
being shaded unless `shadowIncludeSlf4j` is explicitly specified.
Currently we only do this in the kudu-subrocess module.
Change-Id: I72605594bdc70670f82eff4154247c5c0252de7f
Reviewed-on: http://gerrit.cloudera.org:8080/16453
Tested-by: Grant Henke <granthe...@apache.org>
Reviewed-by: Grant Henke <granthe...@apache.org>
> kudu-spark2 jar is now including slf4j classes
> ----------------------------------------------
>
> Key: KUDU-3157
> URL: https://issues.apache.org/jira/browse/KUDU-3157
> Project: Kudu
> Issue Type: Bug
> Components: spark
> Affects Versions: 1.12.0
> Reporter: Jeff Evans
> Assignee: Grant Henke
> Priority: Major
>
> It seems that sometime between versions {{1.7.0}} and {{1.12.0}} of
> {{org.apache.kudu:kudu-spark2_2.11}}, some SLF4J classes started being
> included.
> {code}
> # for 1.7.0
> jar tvf
> ~/.m2/repository/org/apache/kudu/kudu-spark2_2.11/1.7.0/kudu-spark2_2.11-1.7.0.jar
> | grep slf4j
> 1203 Wed Jun 29 14:43:04 CDT 2016 META-INF/license/LICENSE.slf4j.txt
> #for 1.12.0
> jar tvf
> ~/.m2/repository/org/apache/kudu/kudu-spark2_2.11/1.12.0/kudu-spark2_2.11-1.12.0.jar
> | grep slf4j
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/event/
> 555 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/EventConstants.class
> 9842 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/EventRecodingLogger.class
> 1450 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/Level.class
> 432 Mon Dec 16 22:03:32 CST 2019 org/slf4j/event/LoggingEvent.class
> 2615 Mon Dec 16 22:03:32 CST 2019
> org/slf4j/event/SubstituteLoggingEvent.class
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/helpers/
> 3285 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMarker.class
> 1634 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMarkerFactory.class
> 1233 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMDCAdapter$1.class
> 2618 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/BasicMDCAdapter.class
> 1091 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/FormattingTuple.class
> 4773 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/MarkerIgnoringBase.class
> 7553 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/MessageFormatter.class
> 823 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NamedLoggerBase.class
> 3267 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPLogger.class
> 584 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPLoggerFactory.class
> 1249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/NOPMDCAdapter.class
> 10262 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/SubstituteLogger.class
> 2199 Mon Dec 16 22:03:32 CST 2019
> org/slf4j/helpers/SubstituteLoggerFactory.class
> 196 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/Util$1.class
> 765 Mon Dec 16 22:03:32 CST 2019
> org/slf4j/helpers/Util$ClassContextSecurityManager.class
> 2952 Mon Dec 16 22:03:32 CST 2019 org/slf4j/helpers/Util.class
> 180 Mon Dec 16 22:03:32 CST 2019 org/slf4j/ILoggerFactory.class
> 272 Mon Dec 16 22:03:32 CST 2019 org/slf4j/IMarkerFactory.class
> 1375 Mon Dec 16 22:03:32 CST 2019 org/slf4j/Logger.class
> 12575 Mon Dec 16 22:03:32 CST 2019 org/slf4j/LoggerFactory.class
> 666 Mon Dec 16 22:03:32 CST 2019 org/slf4j/Marker.class
> 1621 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MarkerFactory.class
> 177 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC$1.class
> 749 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC$MDCCloseable.class
> 3675 Mon Dec 16 22:03:32 CST 2019 org/slf4j/MDC.class
> 0 Mon Apr 27 14:53:30 CDT 2020 org/slf4j/spi/
> 455 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/LocationAwareLogger.class
> 249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/LoggerFactoryBinder.class
> 249 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/MarkerFactoryBinder.class
> 529 Mon Dec 16 22:03:32 CST 2019 org/slf4j/spi/MDCAdapter.class
> 0 Mon Apr 27 14:53:30 CDT 2020 META-INF/maven/org.slf4j/
> 0 Mon Apr 27 14:53:30 CDT 2020 META-INF/maven/org.slf4j/slf4j-api/
> 3835 Mon Dec 16 21:59:52 CST 2019 META-INF/maven/org.slf4j/slf4j-api/pom.xml
> 109 Mon Dec 16 22:00:08 CST 2019
> META-INF/maven/org.slf4j/slf4j-api/pom.properties
> {code}
> These should be removed from the jar, or at least relocated so as not to
> clash with Spark vendor dependencies, which can cause {{LinkageError}} in
> Spark clusters. Real life example of this problem, seen on a Spark job
> started on CDH 6.3 cluster:
> {code}
> Exception in thread "main" java.lang.LinkageError: loader constraint
> violation: when resolving method
> "org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;"
> the class loader (instance of
> org/apache/spark/util/ChildFirstURLClassLoader) of the current class,
> org/slf4j/LoggerFactory, and the class loader (instance of
> sun/misc/Launcher$AppClassLoader) for the method\'s defining class,
> org/slf4j/impl/StaticLoggerBinder, have different Class objects for the type
> org/slf4j/ILoggerFactory used in the signature
> at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
