[
https://issues.apache.org/jira/browse/FLINK-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283096#comment-16283096
]
ASF GitHub Bot commented on FLINK-7928:
---------------------------------------
Github user shuai-xu commented on a diff in the pull request:
https://github.com/apache/flink/pull/4991#discussion_r155708446
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/types/ResourceProfile.java
---
@@ -61,16 +80,26 @@
* @param heapMemoryInMB The size of the heap memory, in megabytes.
* @param directMemoryInMB The size of the direct memory, in megabytes.
* @param nativeMemoryInMB The size of the native memory, in megabytes.
+ * @param memoryForInputInMB The size of the memory for input, in
megabytes.
+ * @param memoryForOutputInMB The size of the memory for output, in
megabytes.
*/
public ResourceProfile(
double cpuCores,
int heapMemoryInMB,
int directMemoryInMB,
- int nativeMemoryInMB) {
+ int nativeMemoryInMB,
+ int memoryForInputInMB,
+ int memoryForOutputInMB,
--- End diff --
I think resource spec contains the resource user need to run their code,
while resource profile contains the resource for running a task. So resource
profile should also contain the part of resource used by flink system. We
divide these part of resource to memoryForInputInMB and memoryForOutputInMB,
and separate them from heap memory and direct memory so as to different
resource managers can choose different strategies. For example, per job
resource manager need all these resource when allocating a task manager. but
session manager may not consider the memoryForInputInMB and memoryForOutputInMB
when assign a slot, as these part is decide when the session cluster is
created. Do you think it make sense?
> Extend the filed in ResourceProfile for precisely calculating the resource of
> a task manager
> --------------------------------------------------------------------------------------------
>
> Key: FLINK-7928
> URL: https://issues.apache.org/jira/browse/FLINK-7928
> Project: Flink
> Issue Type: Improvement
> Components: JobManager, ResourceManager
> Reporter: shuai.xu
> Assignee: shuai.xu
> Labels: flip-6
>
> ResourceProfile records all the resource requirements for a slot。It is
> generated by JobMaster and then passed to ResourceManager with the slot
> request.
> A task in the slot needs three parts of resource:
> 1. The resource for the operators, this is specified by the ResourceSpec user
> defined
> 2. The resource for the operators to communicating with their upstreams. For
> example, the resource for buffer pools and so on.
> 3. The resource for the operators to communicating with their downstreams.
> Same as above.
> So ResourceProfile should contain three parts of resource, the first part
> from ResouceSpec, and the other two part be generated by Job Master.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
