[
https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184035#comment-17184035
]
Chesnay Schepler commented on FLINK-18695:
------------------------------------------
Am I assuming correctly that you set the number of heap arenas to the same
value as direct arenas? Is my understanding of the results correct that this
doubles the memory footprint (if not, how should the {{Total Number of Chunks}}
row be interpreted)?
>From the SslHandler source, it appears that it does not quite care whether we
>set PREFER_DIRECT to true or false. This is basically just the default for
>usages of AbstractByteBufAllocator#buffer(); the {{SslHandler}} has a separate
>{{wantsDirectBuffer}} flag that is dependent on the {{SslEngineType}}, and for
>the JDK this is always set to false. There are also some cases where the
>{{SslHandler}} explicitly calls {{AbstractByteBufAllocator#heapBuffer()}}.
So here's a wild idea: what if we change {{NettyBufferPool#heapBuffer()}} to
just allocate a direct buffer instead? I essentially want to unblock the
upgrade from allowing heap buffers, and would rather tackle that as a follow-up
memory usage optimization.
> Allow NettyBufferPool to allocate heap buffers
> ----------------------------------------------
>
> Key: FLINK-18695
> URL: https://issues.apache.org/jira/browse/FLINK-18695
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Chesnay Schepler
> Assignee: Yun Gao
> Priority: Major
> Fix For: 1.12.0
>
>
> in 4.1.43 netty made a change to their SslHandler to always use heap buffers
> for JDK SSLEngine implementations, to avoid an additional memory copy.
> However, our {{NettyBufferPool}} forbids heap buffer allocations.
> We will either have to allow heap buffer allocations, or create a custom
> SslHandler implementation that does not use heap buffers (although this seems
> ill-adviced?).
> /cc [~sewen] [~uce] [~NicoK] [~zjwang] [~pnowojski]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
