Very useful, thanks a lot. Regards, Alexis.
Am Di., 21. Feb. 2023 um 12:04 Uhr schrieb Weihua Hu <[email protected] >: > Hi Alexis, > > The maximum Non-Heap is the sum of the memory pool (which is non-hep) max > size. There are 3 memory pools(based on jdk11): > 1. Metaspace, we can control the size with JVM parameter -XX:MaxMetaspaceSize > or Flink configuration: jobmanager.memory.jvm-metaspace.size. For your > job, this pool size is 150m > 2. Compressed Class Space, this is controlled by JVM parameter > -XX:CompressedClassSpaceSize. The default value is 1G or MetaspaceSize - 2 > * InitialBootClassLoaderMetaspaceSize(default is 4194304). For your job, > this pool size is 142m > 3. CodeCache(codeHeap profiled nmethods/non-profiled > nmethods/non-nmethods), this is controlled by JVM parameter > -XX:ReservedCodeCacheSize. The default value is 240m > > So, the maximum non-heap is 150+142+240 = 532m. > > > Best, > Weihua > > > On Tue, Feb 21, 2023 at 2:33 PM Alexis Sarda-Espinosa < > [email protected]> wrote: > >> Hi Weihua, >> >> Thanks for your response, I am familiar with those calculations, the one >> I don't understand is the Maximum Non-Heap value. >> >> Regards, >> Alexis. >> >> On Tue, 21 Feb 2023, 04:45 Weihua Hu, <[email protected]> wrote: >> >>> Hi, Alexis >>> >>> 1. With those configuration, Flink will set JVM parameters -Xms and -Xmx >>> to 673185792(642m),-XX:MaxDirectMemorySize to >>> 67108864(64m),-XX:MaxMetaspaceSize >>> to 157286400(150m), you can find more information from [1] >>> 2. As the hint in Flink UI: "The maximum heap displayed might differ >>> from the configured values depending on the used GC algorithm for this >>> process.", This[2] shows how JVM calculate the max heap memory from >>> configured -Xms/-Xmx >>> >>> >>> [1] >>> https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/memory/mem_setup_jobmanager/ >>> [2] >>> https://stackoverflow.com/questions/52980629/runtime-getruntime-maxmemory-calculate-method >>> >>> Best, >>> Weihua >>> >>> >>> On Tue, Feb 21, 2023 at 12:15 AM Alexis Sarda-Espinosa < >>> [email protected]> wrote: >>> >>>> Hello, >>>> >>>> I have configured a job manager with the following settings (Flink >>>> 1.16.1): >>>> >>>> jobmanager.memory.process.size: 1024m >>>> jobmanager.memory.jvm-metaspace.size: 150m >>>> jobmanager.memory.off-heap.size: 64m >>>> jobmanager.memory.jvm-overhead.min: 168m >>>> jobmanager.memory.jvm-overhead.max: 168m >>>> jobmanager.memory.enable-jvm-direct-memory-limit: "true" >>>> >>>> However, when I look at the job manager dashboard in the UI, I see that >>>> the value of Non-Heap Maximum is reported as 532 MB. Could someone clarify >>>> how this value is calculated? >>>> >>>> In case it's relevant, the effective configuration for JVM Heap is >>>> reported as 642 MB, with the reported maximum being 621 MB. >>>> >>>> Regards, >>>> Alexis. >>>> >>>>
