Hi,
I submitted patches for the pluggable CommitLog. Appreciate any comments.
Thanks!
https://issues.apache.org/jira/browse/CASSANDRA-14062
Rei Odaira
2017-11-06 17:14 GMT-06:00 大平怜 :
> Thanks for the feedback, Ariel,
>
> Based on your comments, we are revisiting our code changes,
> and then
Thanks for the feedback, Ariel,
Based on your comments, we are revisiting our code changes,
and then we will submit a patch for review.
I hope this effort will help further modularize Cassandra
for better maintainability.
Thanks,
Rei Odaira
2017-11-06 8:20 GMT-06:00 Ariel Weisberg :
> Hi,
>
>
Hi,
OK sorry I am very late to the discussion. I think the existing
consensus around doing it is fine I just think you will find that making
the commit log pluggable might be a little trickier than making a cache
which is a glorified K/V store pluggable.
The commit log reaches into a bunch of ot
Sorry, not "rsync" but "fsync"
Regards,
Rei Odaira
2017-11-02 15:57 GMT-05:00 大平怜 :
> Hello Michael,
>
> The page size of the flash is 4KiB. I have to ask someone else about
> the exact specification of the write endurance, but we have two products;
> as a backend flash device, one uses Sa
Hell Ariel,
About the pluggability, we have discussed this topic in the dev list last
May:
https://www.mail-archive.com/dev@cassandra.apache.org/msg11102.html
I don't think the whole community has reached a consensus, but
the result of the discussion at that time was that
1) We were going to rele
Hello Michael,
The page size of the flash is 4KiB. I have to ask someone else about
the exact specification of the write endurance, but we have two products;
as a backend flash device, one uses Samsung's PM963 M.2 NVMe SSD,
and the other uses FibreChannel-attached IBM FlashSystem 840/900.
Each w
Hi,
Just so I don't seem too negative, what I would really like to see is an
in tree implementation. The real challenge there is that the hardware is
not widely available. If it were something you could get in GCE or AWS
or at least get via an emulator that would be a different story.
Ariel
On W
Hi,
OK. It makes sense that most of the existing plumbing is not applicable
since it operates on a filesystem.
How does replay work? Presumably you will need to refactor
CommitLogReplayer as well?
I think the best way for us to decide whether it's something we want in
tree is to see a patch. You
Awesome!! You're two steps ahead ;)
Not sure if you're allowed to share, but can you highlight any details on
endurance and performance? Are the pages 4kb or 16kb? How many writes do you
expect to handle over a 1 year window of the device? I assume because you're
directly accessing the hardware
Hi Michael,
Yes, testing is always a problem, and that is exactly why we would like to
release
our code as a plugin, outside of the main source tree, so that the project
won't
need to test the hardware-dependent code.
The pluggable CommitLog will allow this approach.
Actually, we have already rel
Rei:
One thing that comes up when these type of conversations occur is how the
project can test hardware dependent code. In the case of the PPC64 stuff,
hardware actually got donated to the ASF so Jenkins runs could be done to check
that things work. Any thoughts on this aspect? Might be a bit
Hi Ariel,
CommitLogSegment assumes commit log files stored on a regular file system.
Our CAPI Flash system bypasses OS and directly accesses flash,
so we cannot use the current framework of CommitLogSegment as it is.
Intel's SPDK also bypasses a file system, so we think this kind of
requirement
is
P.S. I am excited to hear about what new hardware can do in this area
compared to existing non-volatile write caches.
On Tue, Oct 31, 2017, at 04:38 PM, Ariel Weisberg wrote:
> Hi,
>
> There are pluggable elements to the commit log such as those used to
> support mmap or compressed.
>
> Can you
I think pluggable commit log is a great idea if we can do it right! 😃
*Especially* given the intent here seems to be motivated by improving endurance
and performance on the given storage you're running C* on. We've actually been
looking at commit log a bunch recently and doing a pretty deep dive
Hi,
There are pluggable elements to the commit log such as those used to
support mmap or compressed.
Can you describe at a high level what a new implementation would look
like and why it can't be a mode of the existing implementation?
You are not proposing changing the format correct?
Regards,
15 matches
Mail list logo
