Subject: Re: Memory constrains running Flink on Kubernetes
Btw, with regard to:
> The default writer-buffer-number is 2 at most for each column family, and the
> default write-buffer-memory size is 4MB.
This isn't what I see when looking at the OPTIONS-XX file in the rocksdb
dir
from
>>> my experience this part of memory would not occupy too much only if you
>>> have many open files.
>>>
>>> Last but not least, Flink would enable slot sharing by default, and even
>>> if you only one slot per taskmanager, there might exists m
g by default, and even
>> if you only one slot per taskmanager, there might exists many RocksDB
>> within that TM due to many operator with keyed state running.
>>
>> Apart from the theoretical analysis, you'd better to open RocksDB native
>> metrics or track the memory
Best
> Yun Tang
> ----------
> *From:* wvl
> *Sent:* Thursday, July 25, 2019 17:50
> *To:* Yang Wang
> *Cc:* Yun Tang ; Xintong Song ;
> user
> *Subject:* Re: Memory constrains running Flink on Kubernetes
>
> Thanks for all the answers so far.
>
>
the memory usage of pods through Prometheus with k8s.
Best
Yun Tang
From: wvl
Sent: Thursday, July 25, 2019 17:50
To: Yang Wang
Cc: Yun Tang ; Xintong Song ; user
Subject: Re: Memory constrains running Flink on Kubernetes
Thanks for all the answers so far.
Espec
e-oom-behavior
>> [2]
>> https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#indexes-and-filter-blocks
>> [3]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/config.html#rocksdb-native-metrics
>>
>> Best
>> Yun Tang
>>
i/Memory-usage-in-RocksDB#indexes-and-filter-blocks
> [3]
> https://ci.apache.org/projects/flink/flink-docs-release-1.8/ops/config.html#rocksdb-native-metrics
>
> Best
> Yun Tang
> --
> *From:* Xintong Song
> *Sent:* Wednesday, July 24, 2019 11:59
&g
___
From: Xintong Song
Sent: Wednesday, July 24, 2019 11:59
To: wvl
Cc: user
Subject: Re: Memory constrains running Flink on Kubernetes
Hi,
Flink acquires these 'Status_JVM_Memory' metrics through the MXBean library.
According to MXBean document, non-heap is "the Java virtual machin
Hi,
Flink acquires these 'Status_JVM_Memory' metrics through the MXBean
library. According to MXBean document, non-heap is "the Java virtual
machine manages memory other than the heap (referred as non-heap memory)".
Not sure whether that is equivalent to the metaspace. If the
'-XX:MaxMetaspaceSize
