Thanks for the feedback, Ariel, Based on your comments, we are revisiting our code changes, and then we will submit a patch for review. I hope this effort will help further modularize Cassandra for better maintainability.
Thanks, Rei Odaira 2017-11-06 8:20 GMT-06:00 Ariel Weisberg <ar...@weisberg.ws>: > Hi, > > OK sorry I am very late to the discussion. I think the existing > consensus around doing it is fine I just think you will find that making > the commit log pluggable might be a little trickier than making a cache > which is a glorified K/V store pluggable. > > The commit log reaches into a bunch of other internal API during replay > and even a few at runtime. I think it's a refactor away from > abstracting out those concerns from the concerns of making log records > durable, providing notifications of durability, and then releasing them > though. > > If I'm the only person unhappy about breaking the plugins in bug fix > releases and the resulting problems that creates for anyone who wants to > operate this hardware in production then I am willing to look past it. > We could also address it via documentation the plugin link page as well > as at landing sites for individual plugins. > > Otherwise I think this is a bigger commitment from us to a larger API > then originally scoped and one that restricts what changes can be made > in bug fix releases to the existing C* commit log. Unless we have > current version and version next of the plugin API so that we can move > ahead in bug fix releases without breaking existing plugins. > > Thanks, > Ariel > > On Thu, Nov 2, 2017, at 04:46 PM, 大平怜 wrote: > > Hell Ariel, > > > > About the pluggability, we have discussed this topic in the dev list last > > May: > > https://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html > > > > I don't think the whole community has reached a consensus, but > > the result of the discussion at that time was that > > 1) We were going to release our extensions as plugins. > > 2) The project would support the plugin ecosystem by creating a plugin > > link > > page on the Web site. > > > > Appreciate it if you could shed new light on the discussion. > > > > > > Thanks, > > Rei Odaira > > > > > > 2017-11-01 17:53 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: > > > > > Hi, > > > > > > Just so I don't seem too negative, what I would really like to see is > an > > > in tree implementation. The real challenge there is that the hardware > is > > > not widely available. If it were something you could get in GCE or AWS > > > or at least get via an emulator that would be a different story. > > > > > > Ariel > > > > > > On Wed, Nov 1, 2017, at 06:46 PM, Ariel Weisberg wrote: > > > > Hi, > > > > > > > > OK. It makes sense that most of the existing plumbing is not > applicable > > > > since it operates on a filesystem. > > > > > > > > How does replay work? Presumably you will need to refactor > > > > CommitLogReplayer as well? > > > > > > > > I think the best way for us to decide whether it's something we want > in > > > > tree is to see a patch. You would need to do this even if it doesn't > > > > make it in tree and you end up having to deploy a patched build. > > > > > > > > Pluggability is a little bit of a touchy subject because we don't > want > > > > to directly or indirectly become responsible for interfaces to out of > > > > tree implementations. I don't know if there is consensus around this, > > > > but I think even if we made the commit log pluggable it would be with > > > > the understanding that we may change the API even in bug fix > releases. > > > > > > > > Down the line where this becomes tricky is unmaintained out of tree > > > > implementations that people depend on being broken due to interface > > > > changes and then no one being around to fix them. People who depend > on > > > > the out of tree implementation have no one to complain to but us. > This > > > > becomes even more likely when the maintainers aren't using the latest > > > > version of C* and are busy with other things. > > > > > > > > You are characterizing the API as being just a few methods on > CommitLog > > > > but that isn't true. > > > > > > > > These are the imports for CommitLogReplayer > > > > > > > > import org.apache.cassandra.concurrent.Stage; > > > > import org.apache.cassandra.concurrent.StageManager; > > > > import org.apache.cassandra.config.CFMetaData; > > > > import org.apache.cassandra.config.Schema; > > > > import org.apache.cassandra.db.*; > > > > import org.apache.cassandra.io.util.FastByteArrayInputStream; > > > > import org.apache.cassandra.io.util.FileUtils; > > > > import org.apache.cassandra.io.util.RandomAccessReader; > > > > import org.apache.cassandra.utils.*; > > > > > > > > And these are the imports for CommitLog > > > > > > > > import org.apache.cassandra.config.Config; > > > > import org.apache.cassandra.config.DatabaseDescriptor; > > > > import org.apache.cassandra.db.*; > > > > import org.apache.cassandra.io.FSWriteError; > > > > import org.apache.cassandra.io.sstable.SSTableDeletingTask; > > > > import org.apache.cassandra.io.util.DataOutputByteBuffer; > > > > import org.apache.cassandra.metrics.CommitLogMetrics; > > > > import org.apache.cassandra.net.MessagingService; > > > > import org.apache.cassandra.service.StorageService; > > > > import org.apache.cassandra.utils.JVMStabilityInspector; > > > > > > > > If we change any code that changes a line in CommitLog or > > > > CommitLogReplayer in a bug fix release it's probably going to break > your > > > > plugin JAR. Anyone running it in production will now have to fix it > and > > > > recompile or be unable to get bug fixes. > > > > > > > > Regards, > > > > Ariel > > > > On Wed, Nov 1, 2017, at 05:25 PM, 大平怜 wrote: > > > > > Hi Ariel, > > > > > > > > > > CommitLogSegment assumes commit log files stored on a regular file > > > > > system. > > > > > Our CAPI Flash system bypasses OS and directly accesses flash, > > > > > so we cannot use the current framework of CommitLogSegment as it > is. > > > > > Intel's SPDK also bypasses a file system, so we think this kind of > > > > > requirement > > > > > is not uncommon. > > > > > > > > > > It would not be easy to reuse AbstractCommitLogSegmentManager, > either, > > > > > because the archiving and synchronization logics have to be > decoupled. > > > > > It would require major rework, and we don't think we should affect > > > > > the existing implementation so much. > > > > > > > > > > We do not change any existing format of CommitLog. Our plugin > will use > > > > > its own format, as it must manage commit logs on the > 4KB-block-oriented > > > > > address spaces of flash devices. > > > > > > > > > > > > > > > Regards, > > > > > Rei Odaira > > > > > > > > > > > > > > > 2017-10-31 15:38 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: > > > > > > > > > > > Hi, > > > > > > > > > > > > There are pluggable elements to the commit log such as those > used to > > > > > > support mmap or compressed. > > > > > > > > > > > > Can you describe at a high level what a new implementation would > look > > > > > > like and why it can't be a mode of the existing implementation? > > > > > > > > > > > > You are not proposing changing the format correct? > > > > > > > > > > > > Regards, > > > > > > Ariel > > > > > > > > > > > > On Tue, Oct 31, 2017, at 04:09 PM, 大平怜 wrote: > > > > > > > Hello, > > > > > > > > > > > > > > We are developing a Cassandra plugin to store CommitLog on our > > > > > > > low-latency > > > > > > > Flash device (CAPI-Flash). To do that, the original CommitLog > > > interface > > > > > > > must be changed to allow plugins. Anyone has any thoughts > about > > > it? We > > > > > > > have our codebase ready, but we think we should start with > > > high-level > > > > > > > discussion. > > > > > > > > > > > > > > The runtime overhead will be minimal. The only overhead will > be > > > changing > > > > > > > method invocations to CommitLog#add(), > > > CommitLog#getCurrentPosition(), > > > > > > > etc. > > > > > > > into interface invocations. > > > > > > > > > > > > > > Synching to CommitLog is one of the performance bottlenecks in > > > Cassandra > > > > > > > especially with batch commit. I think the pluggable CommitLog > > > will allow > > > > > > > other interesting alternatives, such as one using SPDK. > > > Appreciate any > > > > > > > comments. > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > Rei Odaira > > > > > > > > > > > > ------------------------------------------------------------ > > > --------- > > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > --------- > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
