;> >>
> >> >> The design looks good, and I also support leaving space for proposal
> 1.
> >> >>
> >> >> As you know, loading index/filter/data blocks for querying across
> levels
> >> >> would introduce high IO access within the LSM tree
From: Feifan Wang
Sent: Thursday, March 28, 2024 12:35
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-427: Disaggregated State Store
Thanks for your reply, Hangxiang. I totally agree with you about the jni part.
Hi Yun Tang, I just noticed that FLIP-427 mentions “The
uring each filesystem call, that would be N
>> times
>> >> JNI cost compared with the current RocksDB state-backend's JNI cost.
>> >>
>> >> The other question is that the configuration of
>> >> `state.backend.forSt.working-dir` looks too cou
stion is that the configuration of
> >> `state.backend.forSt.working-dir` looks too coupled with the ForSt
> >> state-backend, how would it be if we introduce another disaggregated
> state
> >> storage? Thus, I think `state.backend.disaggregated.working-dir` might
>
question is that the configuration of
>> `state.backend.forSt.working-dir` looks too coupled with the ForSt
>> state-backend, how would it be if we introduce another disaggregated state
>> storage? Thus, I think `state.backend.disaggregated.working-dir` might be a
>> better con
nd, how would it be if we introduce another disaggregated state
> storage? Thus, I think `state.backend.disaggregated.working-dir` might be a
> better configuration name.
>
>
> Best
> Yun Tang
>
>
> From: Hangxiang Yu
> Sent: Wednesday, Marc
us, I think `state.backend.disaggregated.working-dir` might be a
better configuration name.
Best
Yun Tang
From: Hangxiang Yu
Sent: Wednesday, March 20, 2024 11:32
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-427: Disaggregated State Store
Hi, Yue.
Thank
Hi, Yue.
Thanks for the reply.
If we use proposal1, we can easily reuse these optimizations .It is even
> possible to discuss and review the solution together in the Rocksdb
> community.
We also saw these useful optimizations which could be applied to ForSt in
the future.
But IIUC, it's not bindi
Hi Hangxiang,
Thanks for bringing this discussion.
I have a few questions about the Proposal you mentioned in the FLIP.
The current conclusion is to use proposal 2, which is okay for me. My point
is whether we should retain the potential of proposal 1 in the design.
There are the following reason
Hi everyone,
Thanks for your valuable feedback!
Our discussions have been going on for a while.
As a sub-FLIP of FLIP-423 which is nearing a consensus, I would like to
start a vote after 72 hours.
Please let me know if you have any concerns, thanks!
On Mon, Mar 11, 2024 at 11:48 AM Hangxiang Yu
Hi, Jeyhun.
Thanks for the reply.
Is this argument true for all workloads? Or does this argument also hold
for workloads with many small files, which is quite a common case [1] ?
Yes, I think so. The overhead should still be considered negligible,
particularly in comparison to remote I/O, and ot
Hi Hangxiang,
Thanks for the proposal. +1 for it.
I have a few comments.
Proposal 2 has additional JNI overhead, but the overhead is relatively
> negligible when weighed against the latency of remote I/O.
- Is this argument true for all workloads? Or does this argument also hold
for workloads wi
Hi devs,
I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated
State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly
Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang:
- FLIP-427: Disaggregated State Store
This FLIP introduces the initial version o
13 matches
Mail list logo
