Fwd: FsStateBackend,hdfs rpc api too much,FileCreated and FileDeleted is for what?

2019-07-25 Thread 陈Darling
/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java#L799BestYun TangFrom: 陈Darling <chendonglin...@gmail.com>Sent: Tuesday, July 23, 2019 11:32To: qcx978132...@gmail.com <qcx978132...@gmail.com>Cc: user@flink.apache.org <user@flink.apache.org>; myas...@live.com <my

Fwd: FsStateBackend,hdfs rpc api too much,FileCreated and FileDeleted is for what?

2019-07-22 Thread 陈Darling
Hi Yun Tang Your suggestion is very very important to us.  According to your suggestion, We have suggested that users increase the interval time (1 to 5 minutes) and set state.backend.fs.memory-threshold=10k.  But we only have one hdfs cluster, we try to reduce Hdfs api call, I don't know if there

Fwd: FsStateBackend,hdfs rpc api too much,FileCreated and FileDeleted is for what?

2019-07-22 Thread 陈Darling
Hi Yun Tang Your suggestion is very very important to us.  According to your suggestion, We have suggested that users increase the interval time (1 to 5 minutes) and set state.backend.fs.memory-threshold=10k.  But we only have one hdfs cluster, we try to reduce Hdfs api call, I don't know if there

Fwd: FsStateBackend,hdfs rpc api too much,FileCreated and FileDeleted is for what?

2019-07-22 Thread 陈Darling
Hi In my understanding,CreateFile and FileCreated api is different,FileCreated is more like a check api, but I don’t find where it was called in the src source. I don’t understand when  FileCreated Api was called and for what。Is FileCreated api a hdfs internal confirmation api?FLINK-11696  is to re