Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread guo Maxwell
Strongly agree with this point of view that direct IO can bring great benefits. I have reviewed part of the code, and my preliminary judgment is that it is not very common and limited in some situations, for example, it works for commitlog's write path only for this patch.So I suggest that the d

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread Sam
*Glad you brought up compaction here - I think there would be a significant benefit to moving compaction to direct i/o.* +1. Would love to see this get traction. On Mon, 16 Oct 2023 at 19:38, Jon Haddad wrote: > Glad you brought up compaction here - I think there would be a significant > benefi

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread Dinesh Joshi
I haven't looked at the patch yet so take whatever I say here with a pinch of salt. Philosophically, defaults should not change unless there is a clear demonstrable benefit in majority cases for our users. In this case DirectIO should have clear benefits. That said, this is a new feature and I

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread Jon Haddad
Glad you brought up compaction here - I think there would be a significant benefit to moving compaction to direct i/o. On 2023/10/16 16:14:28 Benedict wrote: > I have some plans to (eventually) use the commit log as memtable payload > storage (ie memtables would reference the commit log entries

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread Benedict
I have some plans to (eventually) use the commit log as memtable payload storage (ie memtables would reference the commit log entries directly, storing only indexing info), and to back first level of sstables by reference to commit log entries. This will permit us to deliver not only much bigger

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread Jon Haddad
I haven't looked at the patch, but at a high level, defaulting to direct I/O for commit logs makes a lot of sense to me. On 2023/10/16 06:34:05 "Pawar, Amit" wrote: > [Public] > > Hi, > > CommitLog uses mmap (memory mapped ) segments by default. Direct-IO feature > is proposed through new PR

ci-cassandra 5.0 and trunk now uses in-tree scripts

2023-10-16 Thread Mick Semb Wever
FYI CASSANDRA-18665 has been committed. This makes our ci-cassandra.a.o stage jobs on 5.0 and trunk branches build and test with the in-tree script instead of cassandra-builds. This is a significant change, lots of testing happened, but I'm still expecting some trouble (like everytime we touch ci

New episode of the Apache Cassandra(R) Corner podcast

2023-10-16 Thread Aaron Ploetz
s2e11 - Sarma Pydipally (You may have to download it to play) https://drive.google.com/file/d/1X98HjR0G3wnRaUUuUUvsDyvx05QZYXXN/view?usp=drive_link It will remain in staging for 72 hours, going live (assuming no objections) by Thursday, October 19th. If anyone should have any questions or commen

Re: [DISCUSS] CommitLog default disk access mode

2023-10-16 Thread guo Maxwell
I think what we should do is provide options, and let users make their own decisions. The default behavior should not be modified until some future released version. Pawar, Amit 于2023年10月16日周一 15:51写道: > [Public] > > Hi, > > > > CommitLog uses mmap (memory mapped ) segments by default. Direct