Thank you for your insights. I created (1) to track the work / progress. (1) https://issues.apache.org/jira/browse/CASSANDRA-18830
________________________________________ From: Jon Meredith <jmeredit...@gmail.com> Sent: Thursday, September 7, 2023 15:42 To: dev@cassandra.apache.org Subject: Re: [Discuss] disabling io.netty.transport.noNative in tests NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. I think the Native dependencies were disabled for in-jvm Netty because they prevented the in-jvm dtest InstanceClassLoaders from being garbage collected and were a source of out-of-metaspace exceptions. I'll echo Alex's comment that you will also need to investigate in-jvm upgrade tests. I'm not sure if it's possible to load two different versions of native libraries concurrently. Perhaps the netty code has changed and we can re-enable, or perhaps you can determine what was not being released by the native code -- that would be much better and as Alex says more reflective of the common environment. To check if it is now safe, you can use the ResourceLeakTest - you may have to comment out a few @Ignores - the previous bar was for the looperGossiperNetwork test to complete 100 loops. Jon. On Wed, Sep 6, 2023 at 9:32 AM Alex Petrov <al...@coffeenco.de<mailto:al...@coffeenco.de>> wrote: I think most of the time people actually use netty _with_ native. This might have been introduced when we were tried to make shaded in-JVM dtest jars. If all tests are passing, and we actually do have a confirmtion that native Netty is being used, I would say +1 to remove `noNative`. Just to make sure though, did you have a chance to see if the upgrade tests also work fine? On Thu, Aug 31, 2023, at 1:20 PM, Miklosovic, Stefan wrote: Hi list, Currently, we are skipping the usage of native libraries in Netty as part of testing here (1). In 5.0 branch, we upgraded Netty to 4.1.96 and we brought all native dependencies to the class path so they are there in runtime (x86, arm, mac). I conducted few CI tests for 5.0+ and not having "io.netty.transport.noNative" set to "true" introduces no errors. I think we were just too motivated here to skip stuff left and right. Having this property enabled seems to have no functional effect. Also, one negative side-effect of having this property enabled is that it logs exceptions when running in-jvm-dtests e.g. in IDEA and it pollutes the logs unnecessarily and it is just a visual clutter to deal with every time. To silence this, I set (2) so it skips the logic in (3) completely hence no un-necessary logging will occure. My question is whether we should not remove (4) in 5.0, that means that tests will use native libraries too. That also means that we are running tests closer to a production environment. I just do not see any reason why we should skip this when all tests are just passing with it too with additional benefit of not seeing an exception logged every time when testing it locally. Thanks (1) https://github.com/apache/cassandra-in-jvm-dtest-api/blob/trunk/src/main/java/org/apache/cassandra/distributed/api/ICluster.java#L95-L102<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra-in-jvm-dtest-api%2Fblob%2Ftrunk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fcassandra%2Fdistributed%2Fapi%2FICluster.java%23L95-L102&data=05%7C01%7CStefan.Miklosovic%40netapp.com%7C4f869e92cf9944a9fa2808dbafa85d46%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638296909901595158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=G8PiVroWMG0%2FFpfnvEJ1sEkJeofTGVqBUSQ29dmChWU%3D&reserved=0> (2) https://github.com/apache/cassandra/blob/trunk/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java#L196<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra%2Fblob%2Ftrunk%2Ftest%2Fdistributed%2Forg%2Fapache%2Fcassandra%2Fdistributed%2Fimpl%2FAbstractCluster.java%23L196&data=05%7C01%7CStefan.Miklosovic%40netapp.com%7C4f869e92cf9944a9fa2808dbafa85d46%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638296909901595158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5CeimCMEMkRckfThR%2BKHLnroKFzhE%2B8pcs%2FGBa8SEH0%3D&reserved=0> (3) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/NativeTransportService.java#L163<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra%2Fblob%2Ftrunk%2Fsrc%2Fjava%2Forg%2Fapache%2Fcassandra%2Fservice%2FNativeTransportService.java%23L163&data=05%7C01%7CStefan.Miklosovic%40netapp.com%7C4f869e92cf9944a9fa2808dbafa85d46%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638296909901595158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Bb6mQK6VQGCZk5XCn5lUD2S0QfRSZ5Q2ihJI9Jf88N8%3D&reserved=0> (4) https://github.com/apache/cassandra-in-jvm-dtest-api/blob/trunk/src/main/java/org/apache/cassandra/distributed/api/ICluster.java#L101<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcassandra-in-jvm-dtest-api%2Fblob%2Ftrunk%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fcassandra%2Fdistributed%2Fapi%2FICluster.java%23L101&data=05%7C01%7CStefan.Miklosovic%40netapp.com%7C4f869e92cf9944a9fa2808dbafa85d46%7C4b0911a0929b4715944bc03745165b3a%7C0%7C0%7C638296909901595158%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Te%2FJ1Km80W343q1keTypQsykQvc2R6Ng%2FgAdaugIClQ%3D&reserved=0>
