Of course if is finding the file, you are actively pointing it towards it.
The BashJavaUtils are supposed to use the log4j configuration file /that
is bundled in the BashJavaUtils.jar, /which you are now interfering
with. That's also why it doesn't require all of lib/ to be on the
classpath; the flink-dist jar is enough.
If you configure the log4j configuration file via the env.java.options
in flink-conf.yaml, then it should not be picked up by the BashJavaUtils.
On 30/07/2021 08:48, Yuval Itzchakov wrote:
It is finding the file though, the problem is that the lib/ might not
be on the classpath when the file is being parsed, thus the YAML file
is not recognized as being parsable.
Is there a way to differ the inference from BashJavaUtils to the
actual bootstrap of the app? or perhaps add the lib to the utils
classpath?
On Thu, Jul 29, 2021, 17:55 Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> wrote:
That could be...could you try configuring "env.java.opts:
-Dlog4j.configurationFile=..."?
On 29/07/2021 15:18, Yuval Itzchakov wrote:
Perhaps because I'm passing it as an argument to the job with
-Dlog4j.configurationFile=... ?
On Thu, Jul 29, 2021 at 3:27 PM Chesnay Schepler
<ches...@apache.org <mailto:ches...@apache.org>> wrote:
Looking once more at the stacktrace this happens in the
BashJavaUtils, which only puts flink-dist and not the entire
lib/ directory on the classpath.
What I don't understand is why it would pick up your log4j
file. It should only use the file that is embedded within
BashJavaUtils.jar.
On 29/07/2021 13:11, Yuval Itzchakov wrote:
Hi Chesnay,
So I looked into it, and jackson-databind is definitely in
my UBER jar as a transitive dependency. Additionally, this
uber JAR is added to the `lib/` directly for Flink before I
start the job, yet it still fails.
Are there any other paths I should explore?
One thing to add is that I'm using the
"classloader.resolve-order" = "parent-first" flag due to my
lib having some deps that otherwise cause runtime collision
errors in Kafka.
On Wed, Jul 7, 2021 at 12:25 PM Yuval Itzchakov
<yuva...@gmail.com <mailto:yuva...@gmail.com>> wrote:
Interesting, I don't have bind explicitly on the
classpath, will give it a try.
Although locally it is working properly.
On Wed, Jul 7, 2021, 12:19 Chesnay Schepler
<ches...@apache.org <mailto:ches...@apache.org>> wrote:
According to the log4j documentation you need both
jackson-databind and jackson-dataformat-yaml to be
on the classpath.
Did you make sure that that this is the case in
1.13.1? It could very well be that in 1.9 databind
was on the classpath by chance.
On 07/07/2021 10:22, Arvid Heise wrote:
Hi Yuval,
For some reason the YamlConfigurationFactory is not
correctly loaded and the fallback
XmlConfigurationFactory is used unsuccessfully.
You could try to force DEBUG on that
YamlConfigurationFactory and check for some output like
"Missing dependencies for Yaml support,
ConfigurationFactory {} is inactive"
In particular, it checks for these classes
"com.fasterxml.jackson.databind.ObjectMapper",
"com.fasterxml.jackson.databind.JsonNode",
"com.fasterxml.jackson.core.JsonParser",
"com.fasterxml.jackson.dataformat.yaml.YAMLFactory"
I'm assuming that with improved classloader
separation, you actually need to put your
dependency into `lib/` instead of putting it into
your user jar. But I'm pulling in @Chesnay Schepler
<mailto:ches...@apache.org> who has much more insights.
On Sun, Jul 4, 2021 at 9:45 PM Yuval Itzchakov
<yuva...@gmail.com <mailto:yuva...@gmail.com>> wrote:
Hi,
I am attempting to upgrade Flink from 1.9 to 1.13.1
I am using a YAML based log4j file. In 1.9, it
worked perfectly fine by adding the following
dependency to the classpath (I deploy with an
uber JAR):
"com.fasterxml.jackson.dataformat" %
"jackson-dataformat-yaml" % "2.12.3"
However, with Flink 1.13.1 I get the following
error at runtime (only when running in a
Cluster, running locally works fine):
*flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager [Fatal Error] log4j2.yaml:1:1:
Content is not allowed in prolog.
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager ERROR StatusLogger Error parsing
/opt/foo/conf/log4j2.yaml
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager org.xml.sax.SAXParseException;
systemId: file:///opt/foo/conf/log4j2.yaml;
lineNumber: 1; columnNumber: 1; Content is not
allowed in prolog.*
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
java.xml/com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:261)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
java.xml/com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.xml.XmlConfiguration.<init>(XmlConfiguration.java:92)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.xml.XmlConfigurationFactory.getConfiguration(XmlConfigurationFactory.java:46)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:458)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:385)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:293)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:647)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:668)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:253)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:153)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.LogManager.getContext(LogManager.java:155)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:48)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:30)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:329)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:349)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager at
org.apache.flink.runtime.util.bash.BashJavaUtils.<clinit>(BashJavaUtils.java:45)
flink-aamtr-a11e58230095d9390e8547eccb0290d4-27ef11e7-tm-6r8dfx
taskmanager ERROR StatusLogger No logging
configuration
This indicates that for some reason, the
jackson dataformat YAML library is not getting
properly loaded from my uber JAR at runtime.
Has anyone run into this? Any possible workarounds?
--
Best Regards,
Yuval Itzchakov.
--
Best Regards,
Yuval Itzchakov.
--
Best Regards,
Yuval Itzchakov.
