Anton, Thanks for the write-up. It's very entertaining to read. I left some minor comments on the doc. +1 to the proposal.
Regards, Manu On Tue, Aug 27, 2024 at 7:21 AM Steven Wu <stevenz...@gmail.com> wrote: > Anton, > > Thanks a lot for the improvement proposal and great write-up with > quantitative supporting arguments. > > +1 from my side. > > Thanks, > Steven > > > On Wed, Aug 21, 2024 at 2:29 PM Anton Okolnychyi <aokolnyc...@gmail.com> > wrote: > >> Hey folks, >> >> As discussed during the sync, I've been working on a proposal to improve >> the handling of position deletes in V3. It builds on lessons learned from >> deploying the current approach at scale and addresses all unresolved >> questions from past community discussions and proposals around this topic. >> >> In particular, the proposal attempts to address the following >> shortcomings we observe today: >> >> - Choosing between fewer delete files on disk or targeted deletes. >> - Dependence on external maintenance for consistent write and read >> performance. >> - Writing and reading overhead as in-memory and on-disk >> representations differ. >> >> Please, take a look at the doc [1] and let me know what you think. Any >> feedback is highly appreciated! >> >> - Anton >> >> [1] - >> https://docs.google.com/document/d/18Bqhr-vnzFfQk1S4AgRISkA_5_m5m32Nnc2Cw0zn2XM >> >> >>
