Hi,
I'm afraid there is no nice API way of doing it but you should be able to
access the REST interface of the JobManager to get information about all
TaskManagers. For example on a locally started cluster on my machine I get
this when accessing http://localhost:8081/taskmanagers:
{"taskmanagers":[{"id":"8891a36673c33ac3f77c39b26c6aa516","path":"akka.tcp://flink@192.168.1.134:51310/user/taskmanager","dataPort":51311,"timeSinceLastHeartbeat":1462357320125,"slotsNumber":1,"freeSlots":1,"cpuCores":8,"physicalMemory":17179869184,"freeMemory":536870912,"managedMemory":325351712}]}
With a json parsing library you should be able to extract the memory
characteristics for all TaskManagers.
I hope this helps. :-)
Cheers,
Aljoscha
On Tue, 3 May 2016 at 12:47 Brücke, Christoph <
christoph.brue...@campus.tu-berlin.de> wrote:
> Hi,
>
> I have a problem retrieving `TaskManager` memory configuration settings
> from the `ExecutionEnvironment` or I wonder if there is way at all to do
> so. Basically what I’m trying to do is: I create FlinkJobs using the Java
> DataSet API, for that I also need the available memory on the
> `TaskManager`s, but *before* creating/running the job.
>
> The options I came up with so far, are:
>
> 1. Parse the Flink configuration myself using
> `GlobalConfiguration#loadConfiguration(String)`. This has the disadvantage
> that it doesn’t seem to set default parameters, hence if I use
> `LocalFlinkMiniCluster` or some parameters are simply not set in the YAML
> file they will not be present in the returned `Configuration` object
>
> 2. Create a “dummy” job just to start the `JobManager` and `TaskManager`s
> and somehow access the `TaskManagerConfiguration`, here I don’t see an easy
> way to actually access the `Taskmanager`s configuration.
>
> 3. Use `EnvironmentInformation` to access the memory options from the JVM
> running the Flink `TaskManager`. This has the disadvantage that this only
> gives me a rough estimation on the actual available memory per
> `TaskManager`, since I don’t know how many bytes are reserved for
> `NetworkBuffer`s etc.
>
> Ultimately I only need the the amount of available memory on the
> `TaskManger`s that I can use to store broadcast variables.
>
> Does anyone have an idea how I can retrieve this information ideally
> before actually running a Flink job?
>
> Best regards,
> Christoph
>
>
