Hi All, I went ahead and started an epic to track the work here: https://issues.apache.org/jira/browse/CASSANDRA-20326
Initially the focus will be on removing the driver as a dependency of core server code. Once that has been achieved we can start working on migrating tooling and test code to the 4.x driver. Thanks, Andy On Wed, Feb 12, 2025 at 8:40 PM Tolbert, Andy <x...@andrewtolbert.com> wrote: > > Jon > > > We can probably skip cassandra-stress, since it looks like easy-cass-stress > > can be donated. That does need a driver upgrade to support a vector > > workload, but imo there's no point in investing more in cassandra-stress > > when we have an alternative with more features available. Not a hill I'm > > going to die on, just an opportunity to do less work. > > That sounds great! I think it's likely that we can manage separate > classpaths for the various tools, so we could update the driver dependency in > fqltool and cassandra-loader in the meantime and leave cassandra-stress as is > if it is going to be superseded. > > JD, > > > For the tests, maybe we can have two test class paths for a while? One for > > driver 3 and one for driver 4? That way we don’t need to migrate them all > > in a giant big bang patch? They could be moved over a few at a time making > > review much easier. > > I'll explore if this is possible as I think that could potentially work and > be more manageable. I can also evaluate whether its possible for the two > drivers to co-exist on the same classpath (I think that may be the case, but > I'm not certain). > > Thanks, > Andy > > > On Wed, Feb 12, 2025 at 6:59 PM J. D. Jordan <jeremiah.jor...@gmail.com> > wrote: >> >> Sounds like a reasonable plan to me. +1 >> >> For the tests, maybe we can have two test class paths for a while? One for >> driver 3 and one for driver 4? That way we don’t need to migrate them all >> in a giant big bang patch? They could be moved over a few at a time making >> review much easier. >> >> On Feb 12, 2025, at 6:35 PM, Jon Haddad <j...@rustyrazorblade.com> wrote: >> >> >> Hey Andy, >> >> This seems like a reasonable proposal. >> >> We can probably skip cassandra-stress, since it looks like easy-cass-stress >> can be donated. That does need a driver upgrade to support a vector >> workload, but imo there's no point in investing more in cassandra-stress >> when we have an alternative with more features available. Not a hill I'm >> going to die on, just an opportunity to do less work. >> >> Jon >> >> >> >> >> On Wed, Feb 12, 2025 at 3:06 PM Tolbert, Andy <x...@andrewtolbert.com> wrote: >>> >>> Hi All, >>> >>> I'd like to propose decoupling the java driver as a dependency from the core >>> Cassandra server code. >>> >>> I also want to propose a path towards eventually migrating test and tools >>> code >>> from Apache Cassandra java driver 3.x to 4.x when the time is right for the >>> project. >>> >>> Refactoring test code to 4.x is likely to be quite invasive, as I count >>> 128 source files utilizing driver code. We'd want to find a good time to do >>> this to minimize disruption to ongoing development. >>> >>> Java driver 4.x is effectively a rewrite of the 3.x driver. Its first >>> release >>> was in March of 2019. While it has similar APIs, it is not binary compatible >>> with the 3.x driver [1]. >>> >>> While there hasn't been a clear decision on how the 3.x driver will be >>> supported going forward (although we should consider discussing this!), we >>> expect and have seen active development take place mostly exclusively >>> on the 4.x driver. >>> >>> It would be useful to migrate to the 4.x driver to test new and future >>> features >>> of which the 4.x driver will actively support. For example, the 4.x driver >>> supports Vector types, where the 3.x driver does not. >>> >>> I've iterated the codebase and identified the following uses of the driver: >>> >>> 0. Core code that uses the driver >>> >>> * UntypedResultSet uses CodecUtils.fromUnsignedToSignedInt from the driver >>> which is just adding Integer.MIN_VALUE to an int so can easily be removed. >>> * PreparedStatementHelper is used only by dtest fuzz tests to validate >>> Prepared Statements. Can be moved to test code. >>> * ThreadAwareSecurityManager.checkPermission makes reference to skipping >>> checking accessDeclaredMembers due to use of CodecUtils, can probably >>> remove >>> that with its use removed. >>> * sstableloader uses the driver to fetch schema and metadata >>> >>> 1. Tools that use the driver >>> >>> * fqltool replay (replaying queries from captured logs) >>> * cassandra-stress (making queries to generate load) >>> >>> 2. Test code >>> >>> * Understandably, quite a bit of test code uses the driver. This is where I >>> anticipate the most work would be be needed. >>> >>> I'd like to propose doing the following: >>> >>> Can be done now: >>> >>> * Move sstableloader source into its own tools directly, much like fqltool >>> and cassandra-stress. For compatibility, we could retain the existing >>> shell >>> script entry point (bin/sstableloader). >>> * Update remaining core code to remove all use of the driver. As shown >>> above, >>> there is not much to change here and this should be relatively easy to >>> accomplish. >>> * Update the build and scripts to establish separate classpaths for the >>> server >>> and the respective tools. We would exclude the driver and its >>> dependencies >>> (that aren't required otherwise) from the server. The driver would still >>> be >>> included in the built package, so this wouldn't reduce the size of the >>> binary, but it would remove the driver from the server's classpath, which >>> would de-risk upgrading the driver and having it or its dependencies cause >>> possible runtime issues. >>> >>> To be done next: >>> >>> * Refactor sstableloader, fqltool and cassandra-stress to use the 4.x >>> driver. >>> >>> To be done when the timing works for the project: >>> >>> * Refactor tests to use the 4.x driver. >>> >>> Hopefully this proposed approach makes sense, I'd be eager to hear any >>> feedback or suggestions! >>> >>> Thanks, >>> Andy >>> >>> [1]: >>> https://docs.datastax.com/en/developer/java-driver/4.17/upgrade_guide/index.html#4-0-0
