Hi,

I have a problem retrieving `TaskManager` memory configuration settings from 
the `ExecutionEnvironment` or I wonder if there is way at all to do so. 
Basically what I’m trying to do is: I create FlinkJobs using the Java DataSet 
API, for that I also need the available memory on the `TaskManager`s, but 
*before* creating/running the job.

The options I came up with so far, are:

1. Parse the Flink configuration myself using 
`GlobalConfiguration#loadConfiguration(String)`. This has the disadvantage that 
it doesn’t seem to set default parameters, hence if I use 
`LocalFlinkMiniCluster` or some parameters are simply not set in the YAML file 
they will not be present in the returned `Configuration` object

2. Create a “dummy” job just to start the `JobManager` and `TaskManager`s and 
somehow access the `TaskManagerConfiguration`, here I don’t see an easy way to 
actually access the `Taskmanager`s configuration.

3. Use `EnvironmentInformation` to access the memory options from the JVM 
running the Flink `TaskManager`. This has the disadvantage that this only gives 
me a rough estimation on the actual available memory per `TaskManager`, since I 
don’t know how many bytes are reserved for `NetworkBuffer`s etc.

Ultimately I only need the the amount of available memory on the `TaskManger`s 
that I can use to store broadcast variables.

Does anyone have an idea how I can retrieve this information ideally before 
actually running a Flink job?

Best regards,
Christoph

Reply via email to