Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-19 Thread Runtian Liu
e index, you just need enough info to detect that something >> is missing. If you can detect that view partition x is missing data from >> base partition y, then you could start comparing the actual partition data >> and figure out who’s missing what. >> >> On S

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-18 Thread Runtian Liu
e index files, not the sstable > contents. Reads would still be sequential. > > > Most importantly, I decided against this approach due to the complexity > of ensuring index consistency. Introducing secondary indexes opens up new > challenges, such as keeping them in sync with t

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-17 Thread Runtian Liu
built the merkle tree for each sstable as a storage attached >> index? >> >> Then your repair is merging merkle tables. >> >> >> On Sat, May 17, 2025 at 4:57 PM Runtian Liu wrote: >> >>> > I think you could exploit this to improve your MV

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-17 Thread Runtian Liu
air would compare the intersecting index slices. That would allow you to >> r

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-16 Thread Runtian Liu
<-- >>>> >>>> >>>> The partiton key in the new table is v1. If you simply iterate and >>>> transform and calculate merkle trees on the fly, you'll hit v1=14 with >>>> id=1, but you'll miss id=2 and id=3. You need to g

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-15 Thread Runtian Liu
s, but rather two sets ordered by > different keys. > > I think this is a distinction without a difference. Merkle tree repair > works because the ordering of the data is mostly the same across nodes. > > > On Thu, May 15, 2025 at 9:27 AM Runtian Liu wrote: > >> &g

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-15 Thread Runtian Liu
t; the MV will. That means a subrange repair would have to do a *ton* of IO. > Anyone who's mis-configured a sub-range incremental repair to use too many > ranges will probably be familiar with how long it can take to anti-compact > a bunch of SSTables. With MV sub-range repair, we

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-13 Thread Runtian Liu
t can only contain a single column that is not a primary key >>>column in the base table. >>> >>> At that point what exactly is the value in including anything except the >>> original primary key in the MV's primary key columns unless you are using >>

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-09 Thread Runtian Liu
I’ve added a new section on isolation and consistency . In our current design, materialized-view tables stay eventually consistent,

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-08 Thread Runtian Liu
Here’s my perspective: #1 Accord vs. LWT round trips Based on the insights shared by the Accord experts, it appears that implementing MV using Accord can achieve a comparable number of round trips as the LWT solution proposed in CEP-48. Additionally, it seems that the number of WAN RTTs might be

Re: [DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-06 Thread Runtian Liu
m). > > Another question : What is the frequency of inconsistency detection and > repair for mv and base table ? > > Runtian Liu 于2025年5月7日周三 06:51写道: > >> Hi everyone, >> >> We’d like to propose a new Cassandra Enhancement Proposal: CEP-48: >> Firs

[DISCUSS] CEP-48: First-Class Materialized View Support

2025-05-06 Thread Runtian Liu
Hi everyone, We’d like to propose a new Cassandra Enhancement Proposal: CEP-48: First-Class Materialized View Support . This CEP focuses on addressing the long-standing consistency issues in th

Re: Welcome Jaydeepkumar Chovatia as Cassandra committer

2025-04-30 Thread Runtian Liu
congratulations! On Wed, Apr 30, 2025 at 9:09 AM Francisco Guerrero wrote: > Congrats, Jaydeep! > > On 2025/04/30 11:43:00 Josh McKenzie wrote: > > Hey there Cassandra Devs! > > > > The Apache Cassandra PMC is very happy to announce that Jaydeep Chovatia > has > > accepted the invitation to beco

Re: CEP permission

2025-04-29 Thread Runtian Liu
Awesome, thank you so much! On Tue, Apr 29, 2025 at 9:14 AM C. Scott Andreas wrote: > Hi Runtian, thanks for reaching out. > > Your username should now have permission to create new wiki pages for CEPs > in ASF Confluence. > > Cheers, > > – Scott > > On Apr 29,

CEP permission

2025-04-29 Thread Runtian Liu
Hi, Could you please grant me permission to create a new CEP? My ID is curlylrt. Thank you! Thanks, Runtian

Re: [Committer/reviewer needed] Request for Review of Cassandra PRs

2025-04-21 Thread Runtian Liu
19248 that is waiting for review in case someone >>>> has cycles to review it. >>>> >>>> Reviewing community patches is a great way to learn more about the >>>> codebase and get involved with the project, even if you are not a >>>> co

[Committer/reviewer needed] Request for Review of Cassandra PRs

2025-01-16 Thread Runtian Liu
Hi all, I hope you are doing well. I would like to request your review of two pull requests I've submitted earlier: 1. *Inbound stream throttler* [CASSANDRA− 11303 ] 2. *Clear