Hi All,
Though I have mentioned the -Xms and -Xmx values Flume is consuming all memory
and failing at the end
I have tried adding above parameters in command line as below
a. /usr/hdp/current/flume-server/bin/flume-ng agent -c /etc/flume/conf -f
/etc/flume/conf/flumeSolr.conf -n agent -D
Hi Sri,
Are you using a memory channel? What source/sink?
Can you please paste/link your obfuscated config
What does the metrics endpoint say in terms of channel size, sinkdrainsuccess
etc, for the period leading up to the OOM?
Best,
Iain
Sent from my iPhone
> On Jul 26, 2017, at 8:00 AM, A
Lain,
I am using file channel. Source is spoolDir and Sinks are Solr and HDFS
Please find below my Code
#Flume Configuration Starts
agent.sources = SpoolDirSrc
agent.channels = Channel1 Channel2
agent.sinks = SolrSink HDFSsink
# Configure Source
agent.sources.SpoolDirSrc.channels = Channel1 Ch
Config seems sane.
if you ps auxww|grep -i flume, do you see the java process started with
your Xms/Xmx flags?
I increased the heap & added jmx by adding this to flume-env.sh in the
flume conf dir:
JAVA_OPTS="-Xms2048m -Xmx3072m -Dcom.sun.management.jmxremote"
If you enable jmx you can get some
Iain,
Yes I see the Java process with my Xms/Xmx size like below
//bin/java -Xms2048m -Xmx4048m -Dcom.sun.management.jmxremote -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -Xms1024m -Xmx4048m -cp
/etc/flume/conf:/usr/hdp/2.3.4.0-3485/flume
Let me work on your other suggestions, I will keep you pos
