carp84 commented on issue #9501: [FLINK-12697] [State Backends] Support on-disk state storage for spill-able heap backend URL: https://github.com/apache/flink/pull/9501#issuecomment-550312495 > I'm wondering whether it would be easier to do it now as we are opening the PRs and can update them bit by bit or after everything has been merged. I fear that the inertia might be a bit higher if everything is in place and a much bigger pile of code would needed to be touched. But I'm also ok to do it later if you say that this is not a problem. Thanks for sharing the thoughts @tillrohrmann , I think the concern makes sense. I have been preparing the commit of replacing `ByteBuffer` with `MemorySegment` for some time and have just completed the work. It turns out only one new method required for `MemorySegment` and no modifications for the existing one (we could open another PR if would like to add the hbase-like padding method for operating systems w/o unaligned memory access support, which is not that urgent), so the impact on old components is within control. Please check the new commit for more details. I also checked the integration with the not-committed-yet `SpaceAllocator` using `MemorySegment` and confirmed everything works fine (please check the [PoC branch](https://github.com/carp84/flink/tree/spillable-test) for details), will update the [SpaceAllocator PR](https://github.com/apache/flink/pull/9516) soon if the changes here look good.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
