In general I agree with Joey -- but I would prefer if this behavior is configurable, e.g. there is an option to get a startup failure if the configured fastest provider can't run for any reason to avoid a "silent" performance degradation as Jordan was experiencing.
Thanks, German ________________________________ From: Joseph Lynch <joe.e.ly...@gmail.com> Sent: Thursday, July 20, 2023 7:38 AM To: dev@cassandra.apache.org <dev@cassandra.apache.org> Subject: [EXTERNAL] Re: [DISCUSS] Using ACCP or tc-native by default Having native dependencies shouldn't make the project x86 only, it should just accelerate the performance on x86 when available. Can't we just try to load the fastest available provider (so arm will use native java but x86 will use proper hardware acceleration) and failing that fall-back to the default? If I recall correctly from the messaging service patches (and zstd/lz4) it's reasonably straightforward to try to load native code and then fail-back if you fail. -Joey On Thu, Jul 20, 2023 at 10:27 AM J. D. Jordan <jeremiah.jor...@gmail.com> wrote: > > Maybe we could start providing Dockerfile’s and/or make arch specific rpm/deb > packages that have everything setup correctly per architecture? > We could also download them all and have the startup scripts put stuff in the > right places depending on the arch of the machine running them? > I feel like there are probably multiple ways we could solve this without > requiring users to jump through a bunch of hoops? > But I do agree we can’t make the project x86 only. > > -Jeremiah > > > On Jul 20, 2023, at 2:01 AM, Miklosovic, Stefan > > <stefan.mikloso...@netapp.com> wrote: > > > > Hi, > > > > as I was reviewing the patch for this feature (1), we realized that it is > > not quite easy to bundle this directly into Cassandra. > > > > The problem is that this was supposed to be introduced as a new dependency: > > > > <dependency> > > <groupId>software.amazon.cryptools</groupId> > > <artifactId>AmazonCorrettoCryptoProvider</artifactId> > > <version>2.2.0</version> > > <classifier>linux-x86_64</classifier> > > <dependency> > > > > Notice "classifier". That means that if we introduced this dependency into > > the project, what about ARM users? (there is corresponding aarch classifier > > as well). ACCP is platform-specific but we have to ship Cassandra > > platform-agnostic. It just needs to run OOTB everywhere. If we shipped that > > with x86 and a user runs Cassandra on ARM, I guess that would break things, > > right? > > > > We also can not just add both dependencies (both x86 and aarch) because how > > would we differentiate between them in runtime? That all is just too tricky > > / error prone. > > > > So, the approach we want to take is this: > > > > 1) nothing will be bundled in Cassandra by default > > 2) a user is supposed to download the library and put it to the class path > > 3) a user is supposed to put the implementation of ICryptoProvider > > interface Cassandra exposes to the class path > > 3) a user is supposed to configure cassandra.yaml and its section > > "crypto_provider" to reference the implementation he wants > > > > That way, we avoid the situation when somebody runs x86 lib on ARM or vice > > versa. > > > > By default, NoOpProvider will be used, that means that the default crypto > > provider from JRE will be used. > > > > It can seem like we have not done too much progress here but hey ... we > > opened the project to the custom implementations of crypto providers a > > community can create. E.g. as 3rd party extensions etc ... > > > > I want to be sure that everybody is aware of this change (that we plan to > > do that in such a way that it will not be "bundled") and that everybody is > > on board with this. Otherwise I am all ears about how to do that > > differently. > > > > (1) > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCASSANDRA-18624&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kYGSZGi3caINvm%2FDT4ms3%2BrcnMTxg0E921cMjmUvHQw%3D&reserved=0<https://issues.apache.org/jira/browse/CASSANDRA-18624> > > > > ________________________________________ > > From: German Eichberger via dev <dev@cassandra.apache.org> > > Sent: Friday, June 23, 2023 22:43 > > To: dev > > Subject: Re: [DISCUSS] Using ACCP or tc-native by default > > > > NetApp Security WARNING: This is an external email. Do not click links or > > open attachments unless you recognize the sender and know the content is > > safe. > > > > > > > > +1 to ACCP - we love performance. > > ________________________________ > > From: David Capwell <dcapw...@apple.com> > > Sent: Thursday, June 22, 2023 4:21 PM > > To: dev <dev@cassandra.apache.org> > > Subject: [EXTERNAL] Re: [DISCUSS] Using ACCP or tc-native by default > > > > +1 to ACCP > > > > On Jun 22, 2023, at 3:05 PM, C. Scott Andreas <sc...@paradoxica.net> wrote: > > > > +1 for ACCP and can attest to its results. ACCP also optimizes for a range > > of hash functions and other cryptographic primitives beyond TLS > > acceleration for Netty. > > > > On Jun 22, 2023, at 2:07 PM, Jeff Jirsa <jji...@gmail.com> wrote: > > > > > > Either would be better than today. > > > > On Thu, Jun 22, 2023 at 1:57 PM Jordan West > > <jw...@apache.org<mailto:jw...@apache.org>> wrote: > > Hi, > > > > I’m wondering if there is appetite to change the default SSL provider for > > Cassandra going forward to either ACCP [1] or tc-native in Netty? Our > > deployment as well as others I’m aware of make this change in their fork > > and it can lead to significant performance improvement. When recently > > qualifying 4.1 without using ACCP (by accident) we noticed p99 latencies > > were 2x higher than 3.0 w/ ACCP. Wiring up ACCP can be a bit of a pain and > > also requires some amount of customization. I think it could be great for > > the wider community to adopt it. > > > > The biggest hurdle I foresee is licensing but ACCP is Apache 2.0 licensed. > > Anything else I am missing before opening a JIRA and submitting a patch? > > > > Jordan > > > > > > [1] > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcorretto%2Famazon-corretto-crypto-provider&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bGU%2B7jerm8fiS0Tmav8geFStI%2FcvZKinF35YGKuEQtY%3D&reserved=0<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam04.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Fgithub.com*2Fcorretto*2Famazon-corretto-crypto-provider%26data%3D05*7C01*7CStefan.Miklosovic*40netapp.com*7C4d73ac88a46f4386826e08db742a82f3*7C4b0911a0929b4715944bc03745165b3a*7C0*7C0*7C638231498154472637*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000*7C*7C*7C%26sdata%3DIvtqDSHL*2BOunrhagmYKTfPa8zlknIPijr8TwVvCKKQw*3D%26reserved%3D0__%3BJSUlJSUlJSUlJSUlJSUlJSUlJSUlJQ!!PbtH5S7Ebw!fuCaq60SW0zceq0aFBYG3J8ga4gAtUG5n1qZyiyRSo8X-epQfmmdXqSOXzX4tQbeAMSc5QuTXtz2W0xWyPp59OcTF5Ra%24&data=05%7C01%7CGerman.Eichberger%40microsoft.com%7Cf4530a41df3b419fd2ff08db892f0ed6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638254607439254753%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZWsg5GPkPCDf6EFKcqhEtbYLtEpHtcFbY8ir2i69Nfw%3D&reserved=0<https://github.com/corretto/amazon-corretto-crypto-provider> > > > > > > >
