Hi
In my understanding,CreateFile and FileCreated api is different,FileCreated is more like a check api, but I don’t find where it was called in the src source. I don’t understand when FileCreated Api was called and for what。
Is FileCreated api a hdfs internal confirmation api?
FLINK-11696 is to reduce CreateFile api by reducing mkdir. Will FileCreated Api be reduced?
Is there anything wrong with me here?
Darling Andrew D.Lin
下面是被转发的邮件:
主题: 回复: FsStateBackend,hdfs rpc api too much,FileCreated and FileDeleted is for what?
日期: 2019年7月23日 GMT+8 上午9:48:05
Hi Andrew
These API calls are for checkpoint file created/deleted, and there is an ongoing issue[1] which want to reduce the number.
Hi
We use ‘FsStateBackend' as our state beckend !
The following figure shows the frequency of the hdfs API call.
I don’t understand FilesCreated and FileDeleted is for what? All of these are necessary?
Is it possible to reduce some unnecessary?
|
- Fwd: FsStateBackend,hdfs rpc api too much,FileCreated and FileD... 陈Darling
-