According to the log4j documentation you need both jackson-databind and
jackson-dataformat-yaml to be on the classpath.
Did you make sure that that this is the case in 1.13.1? It could very
well be that in 1.9 databind was on the classpath by chance.
On 07/07/2021 10:22, Arvid Heise wrote:
Hi Yuval,
For some reason the YamlConfigurationFactory is not correctly loaded
and the fallback XmlConfigurationFactory is used unsuccessfully.
You could try to force DEBUG on that YamlConfigurationFactory and
check for some output like
"Missing dependencies for Yaml support, ConfigurationFactory {} is
inactive"
In particular, it checks for these classes
"com.fasterxml.jackson.databind.ObjectMapper",
"com.fasterxml.jackson.databind.JsonNode",
"com.fasterxml.jackson.core.JsonParser",
"com.fasterxml.jackson.dataformat.yaml.YAMLFactory"
I'm assuming that with improved classloader separation, you actually
need to put your dependency into `lib/` instead of putting it into
your user jar. But I'm pulling in @Chesnay Schepler
<mailto:ches...@apache.org> who has much more insights.
On Sun, Jul 4, 2021 at 9:45 PM Yuval Itzchakov <yuva...@gmail.com
<mailto:yuva...@gmail.com>> wrote:
Hi,
I am attempting to upgrade Flink from 1.9 to 1.13.1
I am using a YAML based log4j file. In 1.9, it worked perfectly
fine by adding the following dependency to the classpath (I deploy
with an uber JAR):
"com.fasterxml.jackson.dataformat" % "jackson-dataformat-yaml"
% "2.12.3"
However, with Flink 1.13.1 I get the following error at runtime
(only when running in a Cluster, running locally works fine):
*flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager [Fatal Error] log4j2.yaml:1:1: Content is not allowed
in prolog.
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager ERROR StatusLogger Error parsing /opt/foo/conf/log4j2.yaml
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager org.xml.sax.SAXParseException; systemId:
file:///opt/foo/conf/log4j2.yaml; lineNumber: 1; columnNumber: 1;
Content is not allowed in prolog.*
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
java.xml/com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:261)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
java.xml/com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.xml.XmlConfiguration.<init>(XmlConfiguration.java:92)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.xml.XmlConfigurationFactory.getConfiguration(XmlConfigurationFactory.java:46)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:458)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:385)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:293)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:647)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:668)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.LogManager.getContext(LogManager.java:155)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:48)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:30)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:329)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:349)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.flink.runtime.util.bash.BashJavaUtils.<clinit>(BashJavaUtils.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager ERROR StatusLogger No logging configuration
This indicates that for some reason, the jackson dataformat YAML
library is not getting properly loaded from my uber JAR at runtime.
Has anyone run into this? Any possible workarounds?
--
Best Regards,
Yuval Itzchakov.
