Hi Ariel, CommitLogSegment assumes commit log files stored on a regular file system. Our CAPI Flash system bypasses OS and directly accesses flash, so we cannot use the current framework of CommitLogSegment as it is. Intel's SPDK also bypasses a file system, so we think this kind of requirement is not uncommon.
It would not be easy to reuse AbstractCommitLogSegmentManager, either, because the archiving and synchronization logics have to be decoupled. It would require major rework, and we don't think we should affect the existing implementation so much. We do not change any existing format of CommitLog. Our plugin will use its own format, as it must manage commit logs on the 4KB-block-oriented address spaces of flash devices. Regards, Rei Odaira 2017-10-31 15:38 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: > Hi, > > There are pluggable elements to the commit log such as those used to > support mmap or compressed. > > Can you describe at a high level what a new implementation would look > like and why it can't be a mode of the existing implementation? > > You are not proposing changing the format correct? > > Regards, > Ariel > > On Tue, Oct 31, 2017, at 04:09 PM, 大平怜 wrote: > > Hello, > > > > We are developing a Cassandra plugin to store CommitLog on our > > low-latency > > Flash device (CAPI-Flash). To do that, the original CommitLog interface > > must be changed to allow plugins. Anyone has any thoughts about it? We > > have our codebase ready, but we think we should start with high-level > > discussion. > > > > The runtime overhead will be minimal. The only overhead will be changing > > method invocations to CommitLog#add(), CommitLog#getCurrentPosition(), > > etc. > > into interface invocations. > > > > Synching to CommitLog is one of the performance bottlenecks in Cassandra > > especially with batch commit. I think the pluggable CommitLog will allow > > other interesting alternatives, such as one using SPDK. Appreciate any > > comments. > > > > > > Regards, > > Rei Odaira > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
