Sorry, not "rsync" but "fsync"....
Regards, Rei Odaira 2017-11-02 15:57 GMT-05:00 大平怜 <rei.oda...@gmail.com>: > Hello Michael, > > The page size of the flash is 4KiB. I have to ask someone else about > the exact specification of the write endurance, but we have two products; > as a backend flash device, one uses Samsung's PM963 M.2 NVMe SSD, > and the other uses FibreChannel-attached IBM FlashSystem 840/900. > > Each write is guaranteed to be flushed to the backend flash device > when it returns, so there is no API like rsync. The power loss protection > depends on what the backend flash devices guarantee. For example, > IBM FlashSystem buffers writes in its internal DRAM (as I understand), > but it provides high redundancy in the power units and also includes > batteries. > > In the current implementation, we must write at least 4KiB for each > mutation. > We can change it to merge multiple mutations, much like the batch commit. > > > Thanks, > Rei Odaira > > > 2017-11-01 16:47 GMT-05:00 Michael Kjellman <mkjell...@internalcircle.com> > : > >> Awesome!! You're two steps ahead ;) >> >> Not sure if you're allowed to share, but can you highlight any details on >> endurance and performance? Are the pages 4kb or 16kb? How many writes do >> you expect to handle over a 1 year window of the device? I assume because >> you're directly accessing the hardware as a block device there are >> different rules in regards to fsync and how things are flushed? Any power >> loss protection features etc? If you write a commit log segment that's like >> 20 bytes (for example), will you post-pad the entire thing internally and >> still need to write 4kb (or whatever the physical page size is)? >> >> Thanks! >> >> best, >> kjellman >> >> > On Nov 1, 2017, at 2:40 PM, 大平怜 <rei.oda...@gmail.com> wrote: >> > >> > Hi Michael, >> > >> > Yes, testing is always a problem, and that is exactly why we would like >> to >> > release >> > our code as a plugin, outside of the main source tree, so that the >> project >> > won't >> > need to test the hardware-dependent code. >> > The pluggable CommitLog will allow this approach. >> > >> > Actually, we have already released another plugin for CAPI-Flash-based >> > RowCache, >> > which takes advantage of the pluggable RowCache mechanism. >> > https://github.com/ppc64le/capi-rowcache >> > We would just like to repeat this approach in CommitLog. >> > >> > >> > Thanks, >> > Rei Odaira >> > >> > >> > 2017-11-01 16:30 GMT-05:00 Michael Kjellman < >> mkjell...@internalcircle.com>: >> > >> >> Rei: >> >> >> >> One thing that comes up when these type of conversations occur is how >> the >> >> project can test hardware dependent code. In the case of the PPC64 >> stuff, >> >> hardware actually got donated to the ASF so Jenkins runs could be done >> to >> >> check that things work. Any thoughts on this aspect? Might be a bit >> >> pre-mature, but I thought I'd at least mention it... On the flip side: >> if >> >> CommitLog becomes pluggable enough, shipping an implementation >> compatible >> >> with the hardware out of tree might also be viable too. >> >> >> >> best, >> >> kjellman >> >> >> >>> On Nov 1, 2017, at 2:25 PM, 大平怜 <rei.oda...@gmail.com> wrote: >> >>> >> >>> Hi Ariel, >> >>> >> >>> CommitLogSegment assumes commit log files stored on a regular file >> >> system. >> >>> Our CAPI Flash system bypasses OS and directly accesses flash, >> >>> so we cannot use the current framework of CommitLogSegment as it is. >> >>> Intel's SPDK also bypasses a file system, so we think this kind of >> >>> requirement >> >>> is not uncommon. >> >>> >> >>> It would not be easy to reuse AbstractCommitLogSegmentManager, >> either, >> >>> because the archiving and synchronization logics have to be decoupled. >> >>> It would require major rework, and we don't think we should affect >> >>> the existing implementation so much. >> >>> >> >>> We do not change any existing format of CommitLog. Our plugin will >> use >> >>> its own format, as it must manage commit logs on the >> 4KB-block-oriented >> >>> address spaces of flash devices. >> >>> >> >>> >> >>> Regards, >> >>> Rei Odaira >> >>> >> >>> >> >>> 2017-10-31 15:38 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: >> >>> >> >>>> Hi, >> >>>> >> >>>> There are pluggable elements to the commit log such as those used to >> >>>> support mmap or compressed. >> >>>> >> >>>> Can you describe at a high level what a new implementation would look >> >>>> like and why it can't be a mode of the existing implementation? >> >>>> >> >>>> You are not proposing changing the format correct? >> >>>> >> >>>> Regards, >> >>>> Ariel >> >>>> >> >>>> On Tue, Oct 31, 2017, at 04:09 PM, 大平怜 wrote: >> >>>>> Hello, >> >>>>> >> >>>>> We are developing a Cassandra plugin to store CommitLog on our >> >>>>> low-latency >> >>>>> Flash device (CAPI-Flash). To do that, the original CommitLog >> >> interface >> >>>>> must be changed to allow plugins. Anyone has any thoughts about it? >> >> We >> >>>>> have our codebase ready, but we think we should start with >> high-level >> >>>>> discussion. >> >>>>> >> >>>>> The runtime overhead will be minimal. The only overhead will be >> >> changing >> >>>>> method invocations to CommitLog#add(), >> CommitLog#getCurrentPosition(), >> >>>>> etc. >> >>>>> into interface invocations. >> >>>>> >> >>>>> Synching to CommitLog is one of the performance bottlenecks in >> >> Cassandra >> >>>>> especially with batch commit. I think the pluggable CommitLog will >> >> allow >> >>>>> other interesting alternatives, such as one using SPDK. Appreciate >> any >> >>>>> comments. >> >>>>> >> >>>>> >> >>>>> Regards, >> >>>>> Rei Odaira >> >>>> >> >>>> ------------------------------------------------------------ >> --------- >> >>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> >>>> For additional commands, e-mail: dev-h...@cassandra.apache.org >> >>>> >> >>>> >> >> >> >> >> >> >
