Re: Design doc to fix HDDS-5905

2021-11-08 Thread Kota Uenishi
Important correction: *Thus, the max value of long does not have the first bit as 1.* On Mon, Nov 8, 2021 at 6:49 PM Kota Uenishi wrote: > > Hi Bharat, > > Thank you for the suggestion of object ID. By design, I understand > that object ID is more suitable for delete table use case, regarding > t

Re: Design doc to fix HDDS-5905

2021-11-08 Thread Kota Uenishi
Hi Bharat, Thank you for the suggestion of object ID. By design, I understand that object ID is more suitable for delete table use case, regarding the requirement for monotonicity. I took a glance on HDDS-4315 and I have one question. By looking the code, the object ID seems to have the most sign

Re: Design doc to fix HDDS-5905

2021-11-01 Thread Bharat Viswanadham
Hi Kota, >My question is that, is transaction index always available for non-HA >cluster? Yes, transaction index is available for non-HA also. But when you move from non-HA to non-HA the transaction index starts again from 0, as it is a newly setup cluster and ratis transaction index starts from

Re: Design doc to fix HDDS-5905

2021-10-31 Thread Kota Uenishi
Thank you for the review, Lokesh and Bharat. I understand that transaction id would be better than timestamp, especially because the computation cost of getting timestamp. In this case, requirement for the sorting of deletion keys has not to be strictly monotonic, but just mild monotonicity, like

Re: Design doc to fix HDDS-5905

2021-10-29 Thread Bharat Viswanadham
Hi Kota, Thanks for taking up HDDS-5905 and quickly coming up with a design. I liked the overall approach, but one thing instead of timestamps, I agree with Lokesh, we can use transaction index, and also this will make implementation easy. (As with timestamp, we need to propagate this from the lea

Re: Design doc to fix HDDS-5905

2021-10-28 Thread Lokesh Jain
Hey Kota I really like the proposed approach because it makes sure that blocks are deleted in order of key deletion. I would suggest using Ratis transaction id as the prefix. I don’t think we will need a random suffix with that approach as transaction id would avoid any collisions. Further it a

Design doc to fix HDDS-5905

2021-10-28 Thread Kota Uenishi
Hi Bharat & devs, I've written up some of my idea to fix HDDS-5905, which is a block-leak issue mentioned by Bharat. It involves some data format change in deletion table, so I want to get broader range of feedback from committers in addition to Bharat. If it looks good to you, I want to start wri