[ 
https://issues.apache.org/jira/browse/FLINK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255808#comment-17255808
 ] 

Dian Fu commented on FLINK-18019:
---------------------------------

This problem should still exist and you can verify it before actually digging 
into the problem. 

cc [~dwysakowicz] who may provide more inputs about this issue.

> The configuration specified in TableConfig may not take effect in certain 
> cases
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-18019
>                 URL: https://issues.apache.org/jira/browse/FLINK-18019
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / Legacy Planner, Table SQL / Planner
>    Affects Versions: 1.10.0, 1.11.0
>            Reporter: Dian Fu
>            Priority: Major
>
> Currently If the following configuration is configured in flink-conf.yaml:
> {code:java}
> state.backend: rocksdb
> state.checkpoints.dir: file:///tmp/flink-checkpoints
> {code}
> and the following configuration is configured via TableConfig:
> {code:java}
> tableConfig.getConfiguration().setString("state.backend.rocksdb.memory.fixed-per-slot",
>  "200MB")
> tableConfig.getConfiguration().setString("taskmanager.memory.task.off-heap.size",
>  "200MB")
> {code}
> Then users submit the job via CliFrontend, the configuration set via 
> TableConfig will not take effect.
> Intuitively, it should be that user specified configuration via 
> TableConfig(has higher priority) and the configuration specified via 
> flink-conf.yaml together determines the configuration of a job. However, it 
> doesn't hold in all cases.
> The root cause is that only the configuration specified in TableConfig in 
> passed to *StreamExecutionEnvironment* during translate to plan. For the 
> above case, as *state.backend* is not specified in TableConfig and so the 
> configuration *state.backend.rocksdb.memory.fixed-per-slot* will not take 
> effect. Please note that in above example, the state backend actually used 
> will be RocksDB without the configuration 
> *state.backend.rocksdb.memory.fixed-per-slot* and 
> *taskmanager.memory.task.off-heap.size.*



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to