Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi Yue, Thanks for your comments. I get your point.I think there are two possible ways we can support multiGet in synchronization models in future: 1) Implement the multiGet interface in the existing Rocksdb State interface, then integrate this interface to the synchronous Operators layer. 2) Inte

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

hi Jinzhong Thanks for your reply The reason why I mentioned this point is because according to the official Rocksdb documentation https://rocksdb.org/blog/2022/10/07/asynchronous-io-in-rocksdb.html. if we turn on async_io and use multiGet, it can improve the performance of point look upc by 100%.

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi everyone, Thanks for your valuable discussion and feedback! Our discussions have been going on for a while. If there are no more concerns, I would like to start the vote thread after 72 hours,thanks again! Please let me know if you have any concerns, thanks! Best, Jinzhong On Tue, Mar 19,

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi Yue, Thanks for your feedback! > 1. Does Grouping Remote State Access only support asynchronous interfaces? >--If it is: IIUC, MultiGet can also greatly improve performance for > synchronous access modes. Do we need to support it ？ Yes. If we want to support MultiGet on existing synchrono

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi Jinzhong, Thanks for the FLIP. I have the following questions: 1. Does Grouping Remote State Access only support asynchronous interfaces? --If it is: IIUC, MultiGet can also greatly improve performance for synchronous access modes. Do we need to support it ？ --If not, how can we disti

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

s OOM. > On the other size, if we randomly batch keys, state storage may navigate > all SSTs to find the results. > > Best wishes, > Rui Xia. > > > 发件人: Hangxiang Yu > 发送时间: 2024年3月15日 4:04 > 收件人: xiarui0...@hotmail.com > 主题: Fwd: [

回复: [DISCUSS] FLIP-426: Grouping Remote State Access

results. Best wishes, Rui Xia. 发件人: Hangxiang Yu 发送时间: 2024年3月15日 4:04 收件人: xiarui0...@hotmail.com 主题: Fwd: [DISCUSS] FLIP-426: Grouping Remote State Access -- Forwarded message - From: Jinzhong Li mailto:lijinzhong2...@gmail.com>> Date: Th

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi jeyhun, Thanks for your feedback! >> Do we have a fallback mechanism for filesystems that do not support multiget? Currently, the rocksdb multiGet is available only for PosixFileSystem. And the multiGet requires the filesystem to support async read interfaces (e.g. MultiRead, ReadAsync, poll,

Re: [DISCUSS] FLIP-426: Grouping Remote State Access

Hi Jinzhong, +1 for the FLIP. I have the following comments: - Do we have a fallback mechanism for filesystems that do not support multiget? - Also, in the case of multiget what is the granularity of error handling or retry semantics (e.g., one subrequest fails in multiget). Do we fully rely on

[DISCUSS] FLIP-426: Grouping Remote State Access

Hi devs, I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang: - FLIP-426: Grouping Remote State Access