Hi Alexey, Are you looking to create pull requests for upgrading Netty 4.0 and/or 4.1?
Greg On Thu, May 18, 2017 at 4:41 AM, Alexey Demin <diomi...@gmail.com> wrote: > Hi > > Problem not directly in flink, but it you use flink with beam then in > classpath you have original netty 4.0.27 from flink and netty 4.1.x from > beam (grpc use netty 4.1.x for communication). > > Other interest (specific for me now): netty have custom wrapper for openssl > library which have more productivity versus default jdk version, when you > have enabled ssl cluster communication it's will be very usefull give users > select implementation of ssl (default,openssl,boringssl). > > But netty have a lot of fixes for openssl/boringssl in latest versions and > more preferable do update netty as first step and enable select sslengine > as second step, not all in one step. > > > However, if we do the change at the beginning of the next release cycle, > we > > might have enough exposure time to verify whether things work or not. > > We just start 1.4 iteration and have time for testing. > > Thank, > Alexey Demin > > > 2017-05-18 11:48 GMT+04:00 Till Rohrmann <trohrm...@apache.org>: > > > Hi Alexey, > > > > thanks for looking into it. Are we currently facing any problems with > Netty > > 4.0.27 (bugs or performance)? I agree that in general we should try to > use > > the latest bug fix release. However, in the past we have seen that they > > might entail some slight behaviour changes which breaks things on the > Flink > > side. Since Netty is quite crucial for Flink, I would be extra careful > here > > when bumping versions, especially if there is no strong need for it. > > > > However, if we do the change at the beginning of the next release cycle, > we > > might have enough exposure time to verify whether things work or not. > > > > Cheers, > > Till > > > > On Thu, May 18, 2017 at 8:51 AM, Alexey Demin <diomi...@gmail.com> > wrote: > > > > > Hi > > > > > > For now we use very old netty version. > > > > > > Netty 4.0.27.Final released on 02-Apr-15 > > > > > > <!-- Don't upgrade for now. Netty versions >= 4.0.28.Final > > > contain an improvement by Netty, which slices a Netty buffer > > > instead of doing a memory copy [1] in the > > > LengthFieldBasedFrameDecoder. In some situations, this > > > interacts badly with our Netty pipeline leading to OutOfMemory > > > errors. > > > > > > [1] https://github.com/netty/netty/issues/3704 --> > > > > > > If we so worry about slice in LengthFieldBasedFrameDecoder we can add > > > custom > > > LengthFieldBasedCopyFrameDecoder which extend original > > > LengthFieldBasedFrameDecoder and override extractFrame for keep current > > > behavior. > > > > > > With this small changes we can update for last 4.0.x. > > > > > > For now LengthFieldBasedFrameDecoder also used in KvStateClient and > > > KvStateServer. > > > Can we keep use original LengthFieldBasedFrameDecoder or must also > change > > > on LengthFieldBasedCopyFrameDecoder? > > > > > > If we want we can migrate on 4.1. > > > I already did tests and all work correctly, small changes in > > > NettyBufferPool.java and ChunkedByteBuf.java is required (implement new > > > method which added to interface). > > > > > > > > > Thanks > > > Alexey Diomin > > > > > >
