mdedetrich commented on PR #2409: URL: https://github.com/apache/pekko/pull/2409#issuecomment-3461472543
> > on the `compress` method in `ZstdCompressor` as this will trigger a cleanup on every `onPush` of the stream. I don't know how expensive this cleanup is, there can be an argument that for this specific case we should defer it for GC as GC cleaning up many ByteBuffers as a batch would be faster, @jrudolph can you comment? > > There used to be a reason to use buffer pools especially for direct buffers, probably not least because you might fragment the native heap. Would probably make sense to not churn through them in quick succession. Why not use the buffer pool from the zstd-jni jar? So I just tried doing this and it doesn't work with `ZstdDirectBufferCompressingStreamNoFinalizer`/`ZstdDirectBufferDecompressingStreamNoFinalizer` as they expect direct byte buffers. zstd-jni's `RecyclingBufferPool` is used internally with `ZstdInputStreamNoFinalizer`/`ZstOutputStreamNoFinalizer` which leaves me with 2 options, either 1. Use Pekko's `ByteBufferPool` and rewrite the `ZstdCompressor` logic to deal with fixed size direct `ByteBuffer`'s (shouldn't be too bad) or 2. Use zstd-jni's `ZstdInputStreamNoFinalizer`/`ZstOutputStreamNoFinalizer`, which would likely be simpler but also would be less performant -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
