Hi lorenzo.affetti, Thank you for your feedback. I'm sorry, I didn't fully understand your question. >From the results of the StateBenchmark, it appears that interfaces with regression mainly include （listAdd/listAddAll/mapUpdate/mapRemove/valueAdd/valueUpdate), all of which call Rocksdb.put. So the conclusion is that it will only affect write performance. Of course, I think it is also important and meaningful to investigate the reasons for write performance regression. I will create tickets to track the progress. But investigating the cause will not block our upgrade process this time. I believe that these two can be done in parallel.
Thanks again. <lorenzo.affe...@ververica.com.invalid> 于2024年4月24日周三 15:41写道： > Hello yuema! > > Thank you for the proposal. > In light of what is happening to state backends with FLIP-423 and others, > it definitely makes sense to leverage the full power of latest FRocksDB. > > Small nit and question for you: > do you have any idea how to justify the regression in write perf? > > Thank you again, +1 for this. > On Apr 24, 2024 at 07:58 +0200, Gyula Fóra <gyula.f...@gmail.com>, wrote: > > Thank you for driving this effort > > > > +1 > > > > Cheers > > Gyula > > > > On Wed, 24 Apr 2024 at 06:25, Yuan Mei <yuanmei.w...@gmail.com> wrote: > > > > > Hey Yue, > > > > > > Thanks for all the great efforts significantly improving rescaling and > > > upgrading rocksdb. > > > > > > +1 for this. > > > > > > Best > > > Yuan > > > > > > On Wed, Apr 24, 2024 at 10:46 AM Zakelly Lan <zakelly....@gmail.com> > > > wrote: > > > > > > > > Hi Yue, > > > > > > > > > > Thanks for this proposal! > > > > > > > > > > Given the great improvement we could have, the slight regression > in write > > > > > performance is a worthwhile trade-off, particularly as the > mem-table > > > > > operations contribute only a minor part to the overall overhead. > So +1 > > > for > > > > > this. > > > > > > > > > > > > > > > Best, > > > > > Zakelly > > > > > > > > > > On Tue, Apr 23, 2024 at 12:53 PM Yun Tang <myas...@live.com> > wrote: > > > > > > > > > > > > Hi Yue, > > > > > > > > > > > > > > Thanks for driving this work. > > > > > > > > > > > > > > It has been three years since last major upgrade of FRocksDB. > And it > > > > > would > > > > > > > be great improvement of Flink's state-backend with this > upgrade. +1 for > > > > > > > this work. > > > > > > > > > > > > > > > > > > > > > Best > > > > > > > Yun Tang > > > > > > > ________________________________ > > > > > > > From: Yanfei Lei <fredia...@gmail.com> > > > > > > > Sent: Tuesday, April 23, 2024 12:50 > > > > > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > > > > > Subject: Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 > to 8.10.0 > > > > > > > > > > > > > > Hi Yue & Roman, > > > > > > > > > > > > > > Thanks for initiating this FLIP and all the efforts for the > upgrade. > > > > > > > > > > > > > > 8.10.0 introduces some new features, making it possible for > Flink to > > > > > > > implement some new exciting features, and the upgrade also > makes > > > > > > > FRocksDB easier to maintain, +1 for upgrading. > > > > > > > > > > > > > > I read the FLIP and have a minor comment, it would be better > to add > > > > > > > some description about the environment/configuration of the > nexmark's > > > > > > > result. > > > > > > > > > > > > > > Roman Khachatryan <ro...@apache.org> 于2024年4月23日周二 12:07写道： > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > Thanks for writing the proposal and preparing the upgrade. > > > > > > > > > > > > > > > > > > FRocksDB definitely needs to be kept in sync with the > upstream and > > > the > > > > > > > new > > > > > > > > > APIs are necessary for faster rescaling. > > > > > > > > > We're already using a similar version internally. > > > > > > > > > > > > > > > > > > I reviewed the FLIP and it looks good to me (disclaimer: I > took part > > > in > > > > > > > > > some steps of this effort). > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > Roman > > > > > > > > > > > > > > > > > > On Mon, Apr 22, 2024, 08:11 yue ma <mayuefi...@gmail.com> > wrote: > > > > > > > > > > > > > > > > > > > > Hi Flink devs, > > > > > > > > > > > > > > > > > > > > > > I would like to start a discussion on FLIP-447: > Upgrade FRocksDB > > > from > > > > > > > > > > > 6.20.3 to 8.10.0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-447%3A+Upgrade+FRocksDB+from+6.20.3++to+8.10.0 > > > > > > > > > > > > > > > > > > > > > > This FLIP proposes upgrading the version of FRocksDB > in the Flink > > > > > > > Project > > > > > > > > > > > from 6.20.3 to 8.10.0. > > > > > > > > > > > The FLIP mainly introduces the main benefits of > upgrading FRocksDB, > > > > > > > > > > > including the use of IngestDB which can improve > Rescaling > > > performance > > > > > > > by > > > > > > > > > > > more than 10 times in certain scenarios, as well as > other potential > > > > > > > > > > > optimization points such as async_io, blob db, and > tiered > > > storage.The > > > > > > > > > > > FLIP also presented test results based on RocksDB > 8.10, including > > > > > > > > > > > StateBenchmark and Nexmark tests. > > > > > > > > > > > Overall, upgrading FRocksDB may result in a small > regression of > > > write > > > > > > > > > > > performance( which is a very small part of the overall > overhead), > > > but > > > > > > > it > > > > > > > > > > > can bring many important performance benefits. > > > > > > > > > > > So we hope to upgrade the version of FRocksDB through > this FLIP. > > > > > > > > > > > > > > > > > > > > > > Looking forward to everyone's feedback and > suggestions. Thank you! > > > > > > > > > > > -- > > > > > > > > > > > Best regards, > > > > > > > > > > > Yue > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best, > > > > > > > Yanfei > > > > > > > > > > > > > > > > -- Best, Yue
