ferenc-csaky commented on PR #25866: URL: https://github.com/apache/flink/pull/25866#issuecomment-2595220128
> @ferenc-csaky Yes and No, changing the `io.netty.allocator.type` will affect Flink, too. But the current one I added to Pekko will only affect Pekko. That way, you can keep using the unpooled heap buffer behavior, just like the old Netty 3-based remoting. @He-Pin This is a valid point, although other then Pekko itself Netty is only used in Flink by its shuffle service, which makes it possible to redistribute data between operators in the job pipeline. Having a Pekko-specific option indeed would be useful, but IMO would not worth the trouble for you guys to backport+release it only because of this specific situation, cause according to my understanding it should not cause any problems in real-life scenarios, where memory is not limited into the bare minimum. @normanmaurer Thank you for the Netty side technical details! Enabling reflection definitely not makes any huge difference, but the CI fails sometimes because of a test case, where the memory which is usable by Netty is limited to the bare minimum (7MB), and that causes falkiness, because sometimes that's not even enough to spin up the necessary JVM processes, which actually fails the test, and then the CI. Based on my local test runs, setting `-Dio.netty.tryReflectionSetAccessible=true` seemed to stabilize that specific test. Now I am thinking about adding `-Dio.netty.allocator.type=unpooled` to that specific test as well to make it sure, as the CI is running in a different environment. Personally I believe leaving the Netty4 default settings as is should not cause too much trouble, maybe in some cases, where the memory allocation that Netty can use is kept to the bare minimum with the Netty3 default settings, which may not be true anymore. Anyways, I am okay with moving this discussion to the ML to make sure it reaches a wider audience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
