Thank you for information.
I will investigate my cluster more carefully.
Anyway, at this moment, the WEBUI works well(I don't know why).
If I get some updates, I will inform you.
By the way, I have another problem...
When I run multiple Flink jobs in YARN, the job-logs are mixed.
cf) yarn logs -applicationId <id>
How can I avoid it?
Regards,
Yuta
On 2017/12/06 18:52, Chesnay Schepler wrote:
That said, metrics could still be the issue, with FLINK-7368
<https://issues.apache.org/jira/browse/FLINK-7368> being the prime
suspect. This was fixed for 1.3.3 and 1.4 (both unreleased though).
On 06.12.2017 10:36, Chesnay Schepler wrote:
This can only have an effect if "*taskmanager.net.detailed-metrics*"
is enabled (by default it is disabled).
Otherwise an increase in sub-channels will neither increase the number
of metrics nor the number of polls by the UI.
On 06.12.2017 10:22, Stephan Ewen wrote:
A 'rebalance()' should no impose load on the JM itself. There are
more sub-channels in that case, but the JobManager is not
coordinating anything for them unless there is a failure/recovery.
Each subchannel generates additional metrics, though - might that be
an issue?
On Wed, Dec 6, 2017 at 9:50 AM, Yuta Morisawa
<yu-moris...@kddi-research.jp <mailto:yu-moris...@kddi-research.jp>>
wrote:
> Which version of Fllink are you using?
I'm using Flink 1.3.0.
> Is the high CPU usage caused by the browser or the JVM?
The CPU usage of browser is not high, the JVM is not also.
I found that if I remove the rebalance function in my code, the
WEBUI works well.
I don't know why but I think rebalance function imposes a load on
JobManager.
Regards,
Yuta
On 2017/12/06 17:08, Chesnay Schepler wrote:
Which version of Fllink are you using?
Is the high CPU usage caused by the browser or the JVM?
One thing we should investigate is how many requests the UI
sends to the JobManager,
but I don't know of a good way to do that without modifying
the source.
On 06.12.2017 01 <tel:06.12.2017%2001>:20, Yuta Morisawa wrote:
Hi Timo
I execute streaming job without checkpointing and I don't
configure any state backend, so it may be
"MemoryStateBackend".
Actually, my streaming app just reads data from kafka and
writes it to an external DB. Its not so complicated.
Regards,
Yuta
On 2017/12/05 19:55, Timo Walther wrote:
I had some profiling tool like jvisualvm in mind. Are
you executing streaming or batch jobs? If streaming,
is checkpointing enabled and which type of statebackend?
@Chesnay do you have experience with slow behavior of
the Web UI?
Regards,
Timo
Am 12/5/17 um 10:37 AM schrieb Yuta Morisawa:
Hi Timo
Thank you for your early reply.
These are commands which I run my apps.
./bin/yarn-session.sh -n 20 -jm 6000 -tm 24000 -s 10
./bin/flink run -p 100 <jar1> <jar1 parameters>
./bin/flink run -p 100 <jar2> <jar2 parameters>
So, JobManager Heap Memory = 6000 MB and it
manages 2 jobs.
> Maybe you can use a profiler and find out which
component consumes so
> much CPU resources?
You mean Java Flight Recorder or JITWatch?
Or, Flink has original profiler?
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/application_profiling.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/application_profiling.html>
Regards,
Yuta
On 2017/12/05 18:02, Timo Walther wrote:
Hi Yuta,
as far as I know you cannot assign more cores
to a JobManager.
Can you tell us a bit more about your
environment? How many jobs does the
JobManager has to manage? How much heap
memory is assigned to the JobManager?
Maybe you can use a profiler and find out
which component consumes so much CPU resources?
Regards,
Timo
Am 12/5/17 um 5:13 AM schrieb Yuta Morisawa:
Hi
Now I am looking for the way to increase
the number of allocated CPU cores because
my JobManagaer WEBUI is very heavy and
sometimes freeze.
I think this is caused by the resource
shortage of JobManager.
How can I increase the number of CPU for
JobManager in YARN mode?
Thanks
Yuta
