Thanks for the reply. Talked with Alex in Slack and I was under the impression deleted rows were handled differently on read.
collect ideas about migration, cutoff date and final deprecation in a > separate thread or a document. +1 I am good bringing a subset back to ease upgrades to 4.0 On Thu, Oct 22, 2020 at 11:42 AM Oleksandr Petrov < oleksandr.pet...@gmail.com> wrote: > These are some great (albeit somewhat hard) questions! > > > * when you drop compact storage and have some SSTables stored in > compact storage and some SSTables stored normally, how do we present data > correctly? > > There is no difference in SSTable format between normal and compact > storage, the difference is only in how Table Metadata is represented. In > other words, in code. > > * does upgradesstables and/or compaction migrate off the compact > storage format into the normal format? > > I think above answers this: the only way to migrate off compact storage is > to change table metadata, and accept the differences mentioned above. > > * how do we know it is safe to remove? > > Maybe if we're explicit about it and say that, for example, by version X > compact storage will not be supported at all, and document all the > pitfalls, people will migrate their applications, one at a time. There's > been a few suggestions about how to mitigate, including client options in > 10857, and introducing "hiding" columns in 15811, and many more that were > discussed but not implemented. > > Hope this won't discourage further discussions, but I suggest we keep this > thread focused on pros/cons of a suggested intermediate solution (bring > back a minimal subset of CS), and collect ideas about migration, cutoff > date and final deprecation in a separate thread or a document. I personally > do not have a full answer to "when it is safe to remove", and it seems like > the best we can do is to create a clear procedure. > > On Thu, Oct 22, 2020 at 6:38 PM Ekaterina Dimitrova <e.dimitr...@gmail.com > > > wrote: > > > Definitely that is not off the table. > > Also, we talked this morning for a plan for additional testing to be > > created as part of CASSANDRA-15588 > > > > On Thu, 22 Oct 2020 at 12:33, David Capwell <dcapw...@gmail.com> wrote: > > > > > I am in favor of bringing it back, but I do feel we should also plan > how > > to > > > get it removed as well. > > > > > > Few examples, would love to see fleshed out > > > > > > * when you drop compact storage and have some SSTables stored in > compact > > > storage and some SSTables stored normally, how do we present data > > > correctly? > > > * does upgradesstables and/or compaction migrate off the compact > storage > > > format into the normal format? > > > * how do we know it is safe to remove? > > > > > > > > > > > > On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova < > > e.dimitr...@gmail.com > > > > > > > wrote: > > > > > > > Hi Alex, > > > > Thanks for bringing this up. > > > > I am with you for returning part of the code back and considering > this > > > as a > > > > bug. > > > > I truly believe it is too late in the release to document changed > > > behavior. > > > > I think this contradicts with the project’s promise for no breaking > > > > changes. This should have been documented in alpha. > > > > Best regards, > > > > Ekaterina > > > > > > > > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov < > > > oleksandr.pet...@gmail.com> > > > > wrote: > > > > > > > > > Since this is an important subject, I thought it also makes sense > to > > > > start > > > > > a mailing list thread. > > > > > > > > > > You may know that in 4.0 there was a plan to drop compact storage > and > > > > > related code. However, there are several behavioural changes > related > > to > > > > > compact storage, and difference in visible behaviour between > "normal" > > > and > > > > > compact tables are larger than most of us have anticipated: we > first > > > > > thought there’ll be only “appearing column” in dense case, but > > there’s > > > > > implicit nulls in clusterings thing, and row vs column deletion > now, > > > TTL, > > > > > and more. > > > > > > > > > > Some of the recent issues on the subject are: CASSANDRA-16048 > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which > > allows > > > to > > > > > ignore these differences. The other one was an attempt to improve > the > > > > user > > > > > experience of anyone still using compact storage: CASSANDRA-15811 > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>. > > > > > > > > > > Easily reproducible differences are: > > > > > > > > > > (1) hidden columns show up, which breaks SELECT * queries > > > > > (2) DELETE v and UPDATE v WITH TTL would result into row removals > in > > > > > non-dense compact tables (CASSANDRA-16069 > > > > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>) > > > > > (3) INSERT allows skipping clusterings, which are filled with nulls > > by > > > > > default. > > > > > > > > > > Some of these are tricky to support, as 15811 has shown. Anyone who > > > might > > > > > want to upgrade to 4.0 while still using compact storage might be > > > > affected > > > > > by being forced into one of these behaviours. > > > > > > > > > > Possible solutions are to document these behaviours, or to bring > > back a > > > > > minimal set of COMPACT STORAGE and keep supporting these in 4.0 > > > > > > > > > > It looks like it is possible to leave some of the functionality > > related > > > > to > > > > > DENSE flag and allow it to be present in 4.0, but only for these > > three > > > > (and > > > > > potential related, however not directly visible) cases. > > > > > > > > > > You can find more details on the subject here: > > > > > https://issues.apache.org/jira/browse/CASSANDRA-16217 > > > > > > > > > > Thank you, > > > > > > > > > > -- Alex > > > > > > > > > > > > > > > > > -- > alex p >
