Re: Proposal to retroactively mark materialized views experimental

So, I'm of the opinion there's a difference between users misusing a well understood feature whose shortcomings are widely discussed in the community, and providing a feature we don't fully understand, have not fully documented the caveats of, let alone discovered all the problems with nor had t

Re: Proposal to retroactively mark materialized views experimental

So for some perspective here, how do users who do not get the guarantees of MV’s implement this on their own? They used logged batches. Pseudo CQL here, but you should get the picture: If they don’t ever update data, they do it like so, and it is pretty safe: BEGIN BATCH INSERT tablea blah INSE

Re: Proposal to retroactively mark materialized views experimental

While many users may apparently be using MVs successfully, the problem is how few (if any) know what guarantees they are getting. Since we aren’t even absolutely certain ourselves, it cannot be many. Most of the shortcomings we are aware of are complicated, concern failure scenarios and aren’t

Re: CREATE INDEX without IF NOT EXISTS when snapshoting

Certainly would make sense and should be trivial. here is where you want to look. Just create a ticket for it and prod here for a reviewer once

Re: Proposal to retroactively mark materialized views experimental

Lots of users are already using MV's, believe it or not in some cases quite effectively and also on older versions which were still exposed to a lot of the bugs that cause inconsistencies. 3.11.1 has come a long way since then and I think with a bit more documentation around the current issues mark

Re: Proposal to retroactively mark materialized views experimental

I never used the word easier. I think it's a hard problem but it should be our problem if we want people to use our database. I have little opinion of if MVs should be made experimental or opt-in. I'd simply discussing the need for this feature (as opposed to ripping it out)

Re: Proposal to retroactively mark materialized views experimental

> > the default behavioural unsafety we opt for by not writing through the > batch log We always write to the local batchlog (unless the MV replica is the local node)... Most of the bugs have been around tombstones and ttls AFAIK. On Tue, Oct 3, 2017 at 3:23 PM, Benedict Elliott Smith <_...@bel

Re: Proposal to retroactively mark materialized views experimental

This link is a helpful segway to another problem with MVs and defaults - the default behavioural unsafety we opt for by not writing through the batch log, opening far more windows for data inconsistency than the algorithm otherwise permits. Without a way to detect or repair these inconsistencie

CREATE INDEX without IF NOT EXISTS when snapshoting

Hi everyone, I came across something that bothers me a lot. I'm using snapshots to backup data from my Cassandra cluster in case something really bad happens (like dropping a table or a keyspace). Exercising the recovery actions from those backups, I discover that the schema put on the file "sche

Re: Proposal to retroactively mark materialized views experimental

To clarify: > > * There's no way to determine if a view is out of sync with the base table. > * If you do determine that a view is out of sync, the only way to fix it > is to drop and rebuild the view. In this case, 'out of sync' means 'you lost data', since the current design + repair should kee

Re: Proposal to retroactively mark materialized views experimental

Nobody debates that it's easier, the debate is over whether or not it's correct (and more importantly, whether or not people realize it's not strictly correct in all edge cases). Users expect correct results. People are literally betting their jobs on it. When you have to manually manage sync betw

Re: [VOTE] Release Apache Cassandra 3.11.1

Does seem that way. Fair enough. +1 On Tue, Oct 3, 2017 at 1:37 PM, Aleksey Yeshchenko wrote: > Seems like the patch isn’t exactly ready to commit, and it might be a > while, unlikely < 24 hours before we can roll another one :\ > > Otherwise would’ve flipped to -1 myself :\ > > — > AY > > On 3

Re: Proposal to retroactively mark materialized views experimental

> > The remaining issues are: > > * There's no way to determine if a view is out of sync with the base table. > * If you do determine that a view is out of sync, the only way to fix it > is to drop and rebuild the view. > * There are liveness issues with updates being reflected in the view. > I ju

Re: Proposal to retroactively mark materialized views experimental

Indeed. Paulo and Zhao did a lot of good work to make the situation less bad. You did some as well. Even I retouched small parts of it - metadata related. I’m sorry if I came off as disrespectful - I didn’t mean to. I’ve seen and I appreciate every commit that went into it. It is however my opi

Re: Proposal to retroactively mark materialized views experimental

Lot's of hard work by folks on MVs and I don't think this proposal is a commentary or reflection on that. What it is, is about signalling to users that this feature has more edge cases and caveats than other tried and true features (like all new features). MVs are still a feature in a "stable" rel

Re: [VOTE] Release Apache Cassandra 3.11.1

Seems like the patch isn’t exactly ready to commit, and it might be a while, unlikely < 24 hours before we can roll another one :\ Otherwise would’ve flipped to -1 myself :\ — AY On 3 October 2017 at 10:11:46, Josh McKenzie (jmcken...@apache.org) wrote: Given there's a patch attached to http

Re: Proposal to retroactively mark materialized views experimental

The remaining issues are: * There's no way to determine if a view is out of sync with the base table. * If you do determine that a view is out of sync, the only way to fix it is to drop and rebuild the view. * There are liveness issues with updates being reflected in the view. On October 3, 2017

Re: Proposal to retroactively mark materialized views experimental

On Tue, Oct 3, 2017 at 5:54 PM, Aleksey Yeshchenko wrote: > There are a couple compromise options here: > > a) Introduce the flag (enalbe_experimental_features, or maybe one per > experimental feature), set it to ‘false’ in the yaml, but have the default be > ‘true’. So that if you are upgrading

Re: Proposal to retroactively mark materialized views experimental

Thanks for bringing this up Kurt, it is a fair point. Given the work that Paulo and Zhao have done to get MV’s in shape, what are the outstanding issues that would warrant making them experimental? > On Oct 3, 2017, at 5:56 AM, kurt greaves wrote: > > And finally, back onto the original top

Re: Proposal to retroactively mark materialized views experimental

There are a couple compromise options here: a) Introduce the flag (enalbe_experimental_features, or maybe one per experimental feature), set it to ‘false’ in the yaml, but have the default be ‘true’. So that if you are upgrading from a previous minor to the next without updating the yaml, you n

Re: Proposal to retroactively mark materialized views experimental

> there was "some" reason that even major changes had to be > squeezed into 3.0 before it was released The TL;DR is: having One Version to Rule Them All forces a slew of changes into majors only, since bumping the MessagingService Version has far-reaching impacts. Reference: https://issues.apache.o

Re: Proposal to retroactively mark materialized views experimental

Well this is all terribly interesting. I was actually going to get some discussion going about this during my talk, which unfortunately didn't happen, but I'll take this opportunity to push my agenda. My 99 cents: *tl;dr: we should probably just focus on not releasing completely broken features in

Re: Proposal to retroactively mark materialized views experimental

> CDC sounds like it is in the same basket, but it already has the > `cdc_enabled` yaml flag which defaults false. I went this route because I was incredibly wary of changing the CL code and wanted to shield non-CDC users from any and all risk I reasonably could. I don't know of any outstanding is

Re: Proposal to retroactively mark materialized views experimental

Just one more data point, but I personally don't feel that disabling new MV creation (or new SAS index creation for that matter) by default _in a patch release_ is terribly nice. There can absolutely be code out there that creates MV/SASI indexes somewhat automatically on some events and it would b

Re: [VOTE] Release Apache Cassandra 3.11.1

Given there's a patch attached to https://issues.apache.org/jira/browse/CASSANDRA-13929, there's little sense in us rolling a release with a known leak just to re-roll another in < 24 hours if we can get a reviewer on that. On Tue, Oct 3, 2017 at 4:11 AM, Steinmaurer, Thomas < thomas.steinmau...@d

RE: [VOTE] Release Apache Cassandra 3.11.1

Jon, I have created the following issue: https://issues.apache.org/jira/browse/CASSANDRA-13929 with what we are seeing memory leak wise + discussing a potential fix. Thomas -Original Message- From: Jon Haddad [mailto:jonathan.had...@gmail.com] On Behalf Of Jon Haddad Sent: Montag, 02.