Re: Merging compaction improvements to 5.0

2025-02-12 Thread C. Scott Andreas
Checking to confirm the specific patches proposed for backport – is it the trunk commit for C-20092 and the open GitHub PR against the 5.0 branch for C-15452 linked below? CASSANDRA-20092: Introduce SSTableSimpleScanner for compaction (committed to trunk) https://github.com/apache/cassandra/commi

Re: Merging compaction improvements to 5.0

2025-02-12 Thread guo Maxwell
Of course, I definitely hope to see it merged into 5.0.x as soon as possible Jordan West 于2025年2月13日周四 10:48写道: > Regarding the buffer size, it is configurable. My personal take is that > we’ve tested this on a variety of hardware (from laptops to large instance > sizes) already, as well as a fe

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Caleb Rackliffe
+1 on making this available in 5.0.xWe don’t have to find a default that’s perfect for every hardware configuration. I could understand an argument around disabling read ahead by default in 5.0, but that’s about it. No reason to withhold the capability from users.On Feb 12, 2025, at 9:36 PM, Tolber

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Tolbert, Andy
The data captured in https://issues.apache.org/jira/browse/CASSANDRA-15452 is really exciting. I would also be interested in seeing these changes brought into 5.0. Thanks, Andy On Wed, Feb 12, 2025 at 8:49 PM Jordan West wrote: > Regarding the buffer size, it is configurable. My personal take

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Jordan West
Regarding the buffer size, it is configurable. My personal take is that we’ve tested this on a variety of hardware (from laptops to large instance sizes) already, as well as a few different disk configs (it’s also been run internally, in test, at a few places) and that it has been reviewed by four

Re: [Discuss] Decoupling java driver dependency from server code; migrate tools and tests to 4.x driver

2025-02-12 Thread Tolbert, Andy
Jon > We can probably skip cassandra-stress, since it looks like easy-cass-stress can be donated. That does need a driver upgrade to support a vector workload, but imo there's no point in investing more in cassandra-stress when we have an alternative with more features available. Not a hill I'm g

Re: Merging compaction improvements to 5.0

2025-02-12 Thread guo Maxwell
What I understand is that there will be some differences in block storage among various cloud platforms. More intuitively, the default read-ahead size will be the same. For example, AWS ebs seems to be 256K, and Alibaba Cloud seems to be 512K(If I remember correctly). Just like 19488, give the tes

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Paulo Motta
Thanks Jon for the additional feedback. I will take a look at the ticket more closely and try to reproduce the claimed improvements on my laptop. If there's no regression in performance, I'm +1 in including this improvement in 5.0. On Wed, Feb 12, 2025 at 7:28 PM Jon Haddad wrote: > > Hey Paulo,

Re: [Discuss] Decoupling java driver dependency from server code; migrate tools and tests to 4.x driver

2025-02-12 Thread J. D. Jordan
Sounds like a reasonable plan to me. +1For the tests, maybe we can have two test class paths for a while?  One for driver 3 and one for driver 4?  That way we don’t need to migrate them all in a giant big bang patch?  They could be moved over a few at a time making review much easier.On Feb 12, 202

Re: [Discuss] Decoupling java driver dependency from server code; migrate tools and tests to 4.x driver

2025-02-12 Thread Jon Haddad
Hey Andy, This seems like a reasonable proposal. We can probably skip cassandra-stress, since it looks like easy-cass-stress can be donated. That does need a driver upgrade to support a vector workload, but imo there's no point in investing more in cassandra-stress when we have an alternative wi

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Jon Haddad
Can you elaborate why? This would be several hundred hours of work and would cost me thousands of $$ to perform. Filesystems and block devices are well understood. Could you give me an example of what you think might be different here? This is already one of the most well tested and documented

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Jon Haddad
Hey Paulo, Great questions. I've tested the patch fairly extensively across a wide variety of AWS hardware types both EBS and not. I believe Dave Capwell tested it using infra he had available. In every case I've looked at, it's been a win, or on NVMe barely a change. The reason for this is tha

Re: Merging compaction improvements to 5.0

2025-02-12 Thread guo Maxwell
I think it should be tested on most cloud platforms(at least aws、azure、gcp) before merged into 5.0 . Just like CASSANDRA-19488. Paulo Motta 于2025年2月13日 周四上午6:10写道: > I'm looking forward to these improvements, compaction needs tlc. :-) > A couple of questions: > > Has this been tested only on EB

[Discuss] Decoupling java driver dependency from server code; migrate tools and tests to 4.x driver

2025-02-12 Thread Tolbert, Andy
Hi All, I'd like to propose decoupling the java driver as a dependency from the core Cassandra server code. I also want to propose a path towards eventually migrating test and tools code from Apache Cassandra java driver 3.x to 4.x when the time is right for the project. Refactoring test code to

Re: Merging compaction improvements to 5.0

2025-02-12 Thread Paulo Motta
I'm looking forward to these improvements, compaction needs tlc. :-) A couple of questions: Has this been tested only on EBS, or also EC2/bare-metal/Azure/etc? My only concern is if this is an optimization for EBS that can be a deoptimization for other environments. Are there reproducible scripts

Merging compaction improvements to 5.0

2025-02-12 Thread Jon Haddad
Hey folks, Over the last 9 months Jordan and I have worked on CASSANDRA-15452 [1]. The TL;DR is that we're internalizing a read ahead buffer to allow us to do fewer requests to disk during compaction and range reads. This results in far fewer system calls (roughly 16x reduction) and on systems wi

[RELEASE] Apache Cassandra Java Driver 4.19.0 released

2025-02-12 Thread Bret McGuire
The Cassandra team is pleased to announce the release of Cassandra Java Driver version 4.19.0. This release includes a large number of bug fixes and improvements representing the work of many contributors. Thanks to everyone who made this release possible! The Source release and Binary conveni