Hi Paul,

For parallelism, it should be able to be set with 
`table.exec.resource.default-parallelism` [1] ,
and an example to set the parameter is at the first several paragraph. 

But Regarding the total process memory, I think it should be only set in the 
cluster level since 
it is per-cluster option: the TM should use the option on startup, before the 
job is submitted.

Best,
Yun



[1] 
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/dev/table/config/#table-exec-resource-default-parallelism




------------------------------------------------------------------
From:Paul Lam <paullin3...@gmail.com>
Send Time:2021 Jul. 16 (Fri.) 12:21
To:user <user@flink.apache.org>
Subject:Set job specific resources in one StreamTableEnvironment

Hi,
I’m reusing the same StreamTableEnvironment to submit multiple table/sql jobs 
to a session cluster, 
but I couldn’t find a proper way to specify job resources for each job (like 
parallelism and total process 
memory), and they all uses the cluster default. 

I have considered overriding resources specs of all nodes in the StreamGraph, 
but it’s problematic 
because some nodes have a parallelism limit (e.g. can’t be greater than 1).
I think I might be missing something and there should be a better way to do 
this. Please give me some 
pointers. Thanks a lot!
Best,
Paul Lam

Reply via email to