Hello Michael, The page size of the flash is 4KiB. I have to ask someone else about the exact specification of the write endurance, but we have two products; as a backend flash device, one uses Samsung's PM963 M.2 NVMe SSD, and the other uses FibreChannel-attached IBM FlashSystem 840/900.
Each write is guaranteed to be flushed to the backend flash device when it returns, so there is no API like rsync. The power loss protection depends on what the backend flash devices guarantee. For example, IBM FlashSystem buffers writes in its internal DRAM (as I understand), but it provides high redundancy in the power units and also includes batteries. In the current implementation, we must write at least 4KiB for each mutation. We can change it to merge multiple mutations, much like the batch commit. Thanks, Rei Odaira 2017-11-01 16:47 GMT-05:00 Michael Kjellman <mkjell...@internalcircle.com>: > Awesome!! You're two steps ahead ;) > > Not sure if you're allowed to share, but can you highlight any details on > endurance and performance? Are the pages 4kb or 16kb? How many writes do > you expect to handle over a 1 year window of the device? I assume because > you're directly accessing the hardware as a block device there are > different rules in regards to fsync and how things are flushed? Any power > loss protection features etc? If you write a commit log segment that's like > 20 bytes (for example), will you post-pad the entire thing internally and > still need to write 4kb (or whatever the physical page size is)? > > Thanks! > > best, > kjellman > > > On Nov 1, 2017, at 2:40 PM, 大平怜 <rei.oda...@gmail.com> wrote: > > > > Hi Michael, > > > > Yes, testing is always a problem, and that is exactly why we would like > to > > release > > our code as a plugin, outside of the main source tree, so that the > project > > won't > > need to test the hardware-dependent code. > > The pluggable CommitLog will allow this approach. > > > > Actually, we have already released another plugin for CAPI-Flash-based > > RowCache, > > which takes advantage of the pluggable RowCache mechanism. > > https://github.com/ppc64le/capi-rowcache > > We would just like to repeat this approach in CommitLog. > > > > > > Thanks, > > Rei Odaira > > > > > > 2017-11-01 16:30 GMT-05:00 Michael Kjellman < > mkjell...@internalcircle.com>: > > > >> Rei: > >> > >> One thing that comes up when these type of conversations occur is how > the > >> project can test hardware dependent code. In the case of the PPC64 > stuff, > >> hardware actually got donated to the ASF so Jenkins runs could be done > to > >> check that things work. Any thoughts on this aspect? Might be a bit > >> pre-mature, but I thought I'd at least mention it... On the flip side: > if > >> CommitLog becomes pluggable enough, shipping an implementation > compatible > >> with the hardware out of tree might also be viable too. > >> > >> best, > >> kjellman > >> > >>> On Nov 1, 2017, at 2:25 PM, 大平怜 <rei.oda...@gmail.com> wrote: > >>> > >>> Hi Ariel, > >>> > >>> CommitLogSegment assumes commit log files stored on a regular file > >> system. > >>> Our CAPI Flash system bypasses OS and directly accesses flash, > >>> so we cannot use the current framework of CommitLogSegment as it is. > >>> Intel's SPDK also bypasses a file system, so we think this kind of > >>> requirement > >>> is not uncommon. > >>> > >>> It would not be easy to reuse AbstractCommitLogSegmentManager, either, > >>> because the archiving and synchronization logics have to be decoupled. > >>> It would require major rework, and we don't think we should affect > >>> the existing implementation so much. > >>> > >>> We do not change any existing format of CommitLog. Our plugin will use > >>> its own format, as it must manage commit logs on the 4KB-block-oriented > >>> address spaces of flash devices. > >>> > >>> > >>> Regards, > >>> Rei Odaira > >>> > >>> > >>> 2017-10-31 15:38 GMT-05:00 Ariel Weisberg <ar...@weisberg.ws>: > >>> > >>>> Hi, > >>>> > >>>> There are pluggable elements to the commit log such as those used to > >>>> support mmap or compressed. > >>>> > >>>> Can you describe at a high level what a new implementation would look > >>>> like and why it can't be a mode of the existing implementation? > >>>> > >>>> You are not proposing changing the format correct? > >>>> > >>>> Regards, > >>>> Ariel > >>>> > >>>> On Tue, Oct 31, 2017, at 04:09 PM, 大平怜 wrote: > >>>>> Hello, > >>>>> > >>>>> We are developing a Cassandra plugin to store CommitLog on our > >>>>> low-latency > >>>>> Flash device (CAPI-Flash). To do that, the original CommitLog > >> interface > >>>>> must be changed to allow plugins. Anyone has any thoughts about it? > >> We > >>>>> have our codebase ready, but we think we should start with high-level > >>>>> discussion. > >>>>> > >>>>> The runtime overhead will be minimal. The only overhead will be > >> changing > >>>>> method invocations to CommitLog#add(), CommitLog#getCurrentPosition() > , > >>>>> etc. > >>>>> into interface invocations. > >>>>> > >>>>> Synching to CommitLog is one of the performance bottlenecks in > >> Cassandra > >>>>> especially with batch commit. I think the pluggable CommitLog will > >> allow > >>>>> other interesting alternatives, such as one using SPDK. Appreciate > any > >>>>> comments. > >>>>> > >>>>> > >>>>> Regards, > >>>>> Rei Odaira > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >>>> For additional commands, e-mail: dev-h...@cassandra.apache.org > >>>> > >>>> > >> > >> > >
