numinnex commented on PR #3068: URL: https://github.com/apache/iggy/pull/3068#issuecomment-4258695204
I've looked through the PR and if I understand the design correctly it trades off disk space savings by not storing padding on each written buffer at the expense of complexity. I am not sure if it's the right trade-off, given that we could use background compaction to rewrite those small padded buffers into a larger contiguous blocks. I think rather than storing the `tail` as leftover from misaligned writes, we should pass collection of already `padded` and `aligned` buffers into the `write_vectored_all` method (I know it's impossible without a lot of changes due to prolific usage of `Bytes`, that's why this feature is so hard to implement), but if this PR meant to be worth merging, without immediately becoming technical debt, we have to get it right from the get go. You can take a lot at the `io_buf` or `iggy_io_buf` (I think this is how it' called now) module. I've created there an refcounted buffer that uses `AVec` as it's underlying storage and see if it's even feasible at all to replace `Bytes` with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
