[
https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167067#comment-17167067
]
Nico Kruber commented on FLINK-18695:
-------------------------------------
(unrelated to this thread: [~sewen] Do we actually also cover the receiving
side and use our own memory segments deep in the Netty receiver stack?)
As for OpenSSL: we only ship binaries of a wrapper that *dynamically links* to
OpenSSL binaries on the OS. That, however, breaks too easily. There is also a
statically-linked variant which includes OpenSSL binaries that should always
work (similarly to RocksDB), but we couldn't use that one yet for licensing
reasons. As soon as OpenSSL 3 is out (Apache 2 licensed!) and Netty has
wrappers for that, we can use OpenSSL by default.
Regarding the heap buffers, I don't have any objection as long as it doesn't
degrade performance - we have
[microbenchmarks|https://github.com/apache/flink-benchmarks/blob/a37619dc64991d6a53536fe0b26120ad67e10298/src/test/java/org/apache/flink/benchmark/StreamNetworkThroughputBenchmarkExecutor.java#L71]
to [look at|http://codespeed.dak8s.net:8000/changes/] and I'd be curious to
see how it performs in my small application benchmark I did for
https://www.ververica.com/blog/how-openssl-in-ververica-platform-improves-your-flink-job-performance
> Allow NettyBufferPool to allocate heap buffers
> ----------------------------------------------
>
> Key: FLINK-18695
> URL: https://issues.apache.org/jira/browse/FLINK-18695
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Chesnay Schepler
> Priority: Major
> Fix For: 1.12.0
>
>
> in 4.1.43 netty made a change to their SslHandler to always use heap buffers
> for JDK SSLEngine implementations, to avoid an additional memory copy.
> However, our {{NettyBufferPool}} forbids heap buffer allocations.
> We will either have to allow heap buffer allocations, or create a custom
> SslHandler implementation that does not use heap buffers (although this seems
> ill-adviced?).
> /cc [~sewen] [~uce] [~NicoK] [~zjwang] [~pnowojski]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
