Hi,
With regards to logging (both Flink & application specific logs) within
the container, are there best practices that you know of to get the logs to
a centralized locations..
For e.g. the flink TM's log are local inside the container and I don't
wish to write to shared/mounted volumes, this means that I have to run
separate daemon running inside the container along with the TM to transport
these logs to another server or so...
+ I don't see that flink provides support for syslog to be able to
connect to rsyslog etc.,
Can you please advice a way to go here?
Best Regards
CVP
On Wed, Mar 29, 2017 at 11:33 AM, Patrick Lucas <patr...@data-artisans.com>
wrote:
> For 1., I think the standard approach would be to specify from without
> what the heap size should be. If you want an *x* MB heap, you could set
> your container memory limit to 1.3 * *x* or so (to account for overhead)
> and set taskmanager.heap.mb: *x* in your config.
>
> The other way around—e.g. from inside the container determine its memory
> limit and divide it by 1.3—sounds interesting though, so please share if
> you have success with that.
>
> For 2. I don't think there's really a good way yet to monitor the health
> of containerized jobs directly, so probably your best bet is to watch the
> job's metrics from outside the Flink cluster.
>
> --
> Patrick Lucas
>
> On Wed, Mar 29, 2017 at 10:58 AM, Chakravarthy varaga <
> chakravarth...@gmail.com> wrote:
>
>> Hi,
>>
>> Any updates here? I'm sure many would have faced similar issues like
>> these, any help here is highly appreciated.
>>
>> Best Regards
>> CVP
>>
>> On Tue, Mar 28, 2017 at 5:47 PM, Chakravarthy varaga <
>> chakravarth...@gmail.com> wrote:
>>
>>> Hi Team,
>>>
>>> If the flink cluster is conainerized and managed through by a
>>> container orchestrator,
>>>
>>> 1. the orchestrator allocates resources for each JM. TM etc., say
>>> if the container (TM) needs to run with 2G RAM, how should this allocation
>>> be honoured by the TM when its JVM starts. I'm thinking of wrapping up a
>>> script that determines the resource allocation for the container and writes
>>> the flink-conf.yaml before the TM starts the process. Is this the way to go?
>>>
>>> 2. The container orchestrator looks at health of the containers and
>>> is however unaware of the job health status runnning inside the
>>> container/cluster. How should this be determined?
>>>
>>> Best Regards
>>> CVP
>>>
>>>
>>>
>>
>
