[
https://issues.apache.org/jira/browse/FLINK-7258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ufuk Celebi updated FLINK-7258:
-------------------------------
Description:
In NettyBootstrap we configure the low and high watermarks in the following
order:
{code}
bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK,
config.getMemorySegmentSize() + 1);
bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 2 *
config.getMemorySegmentSize());
{code}
When the memory segment size is higher than the default high water mark, this
throws an `IllegalArgumentException` when a client tries to connect. Hence,
this unfortunately only happens during runtime when a intermediate result is
requested. This doesn't fail the job, but logs a warning and ignores the failed
configuration attempt, potentially resulting in degraded performance because of
a lower than expected watermark.
A simple fix is to first configure the high water mark and only then configure
the low watermark.
was:
In NettyBootstrap we configure the low and high watermarks in the following
order:
{code}
bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK,
config.getMemorySegmentSize() + 1);
bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 2 *
config.getMemorySegmentSize());
{code}
When the memory segment size is higher than the default high water mark, this
throws an `IllegalArgumentException` when a client tries to connect. Hence,
this unfortunately only fails during runtime when a intermediate result is
requested.
A simple fix is to first configure the high water mark and only then configure
the low watermark.
> IllegalArgumentException in Netty bootstrap with large memory state segment
> size
> --------------------------------------------------------------------------------
>
> Key: FLINK-7258
> URL: https://issues.apache.org/jira/browse/FLINK-7258
> Project: Flink
> Issue Type: Bug
> Components: Network
> Affects Versions: 1.3.1
> Reporter: Ufuk Celebi
> Assignee: Ufuk Celebi
>
> In NettyBootstrap we configure the low and high watermarks in the following
> order:
> {code}
> bootstrap.childOption(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK,
> config.getMemorySegmentSize() + 1);
> bootstrap.childOption(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 2 *
> config.getMemorySegmentSize());
> {code}
> When the memory segment size is higher than the default high water mark, this
> throws an `IllegalArgumentException` when a client tries to connect. Hence,
> this unfortunately only happens during runtime when a intermediate result is
> requested. This doesn't fail the job, but logs a warning and ignores the
> failed configuration attempt, potentially resulting in degraded performance
> because of a lower than expected watermark.
> A simple fix is to first configure the high water mark and only then
> configure the low watermark.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
