Hi, FYI I am planning to put together all we discussed under sstable encryption thread into a kind-of CEP to distil everything said. I hope I'll put it together in the foreseeable future before taking a longer break till February. I am not sure if that is enough time to implement it if we eventually vote on that (which is rather hypothetical at this point) but I would appreciate having something solid to elaborate on.
Regards On Tue, 14 Dec 2021 at 17:41, Benjamin Lerer <ble...@apache.org> wrote: > > Sure :-) > > I will make a separate section. > > Le mar. 14 déc. 2021 à 17:30, C. Scott Andreas <sc...@paradoxica.net> a > écrit : > > > Thanks Benjamin! > > > > I see that the Roadmap doc on Confluence contains several features that > > are large in scope but do not have a published CEP or discussion on the > > mailing list. > > > > Because users will treat the roadmap doc as indicative of the project’s > > direction and intent, can these items be moved to a section indicating that > > there is interest or that they are potential features pending proposal and > > discussion? > > > > I don't think it's a good idea to list features that haven't been > > discussed on a user-visible roadmap page, but wouldn't object to the idea > > of them being in a separate section. > > > > – Scott > > > > On Dec 13, 2021, at 6:42 AM, Benjamin Lerer <ble...@apache.org> wrote: > > > > > > I finally wrote down the roadmap on the wiki, and updated it to reflect the > > current situation ( > > https://cwiki.apache.org/confluence/display/CASSANDRA/Roadmap) > > Sorry for the delay. > > > > Le lun. 24 mai 2021 à 19:44, Ekaterina Dimitrova <e.dimitr...@gmail.com> a > > écrit : > > > > Thanks Paulo! > > > > The patch is available on github. It depends only on our availability and > > priorities. I see Benedict mentioned on the ticket that probably 90% of the > > patch covers already the idea of his proposal. I will be happy to finish > > the work or if I am not available, when the time for it comes to handover > > it to someone else. > > > > I can’t find the related ticket now but one of the jvm prerequisites we > > needed is solved - to be able to load custom types. > > > > > > > > On Wed, 19 May 2021 at 5:04, Benjamin Lerer <ble...@apache.org> wrote: > > > > > Thanks Paulo. That one definitely fell through the cracks. > > > > > > I have been pretty busy lately but as soon as I have a bit of time I will > > > create a roadmap page to > > > summarize everything that was proposed so far. Including Making > > SSLContext > > > creation pluggable proposed by Maulin and the JUnit 5 migration proposal > > > from Aleksei. > > > > > > Le mer. 19 mai 2021 à 01:48, Paulo Motta <pauloricard...@gmail.com> a > > > écrit : > > > > > > > I would love to see Ekaterina's work from CASSANDRA-15234 to > > standardize > > > > configuration be resumed in the next releases. > > > > > > > > Thought it would be worth mentioning since we had quite a productive > > > > discussion before putting it on hold to focus on 4.0, so it would be > > > great > > > > to have that conversation resumed. > > > > > > > > Em seg., 26 de abr. de 2021 às 14:16, Benedict Elliott Smith < > > > > bened...@apache.org> escreveu: > > > > > > > > > I think my earlier response vanished into the moderator queue. Just a > > > few > > > > > comments: > > > > > > > > > > 1) The Paxos latency (and correctness) improvements I think should > > land > > > > in > > > > > 4.0.x, as we have introduced a fairly significant regression and this > > > > work > > > > > mostly resolves outstanding issues with LWTs today. > > > > > 2) If we aim to deliver multi-partition LWTs in 4.x/5.0, we may > > likely > > > > > want to pair this with work to further reduce latency beyond the > > above > > > > > work, as contention will become a more significant problem. Should I > > be > > > > > involved in delivering multi-partition LWTs I will also be aiming to > > > > > deliver even lower latencies for the release they land in. > > > > > 3) To support all of the above work, I also aim to deliver a > > Simulator > > > > > facility for deterministically executing cluster workloads under > > > > > adversarial scheduling (i.e. that intercepts all message and thread > > > > events > > > > > and evaluates them sequentially, in pseudorandom order), alongside > > > > > linearizability verification built upon this. This work will include > > > (or > > > > > have as a prerequisite) significant clean-ups to internal > > functionality > > > > > like executors, use of futures and other concurrency primitives, and > > > > > mocking out of time and the filesystem. > > > > > > > > > > > > > > > On 23/04/2021, 14:46, "Benjamin Lerer" <b.le...@gmail.com> wrote: > > > > > > > > > > Hi everybody, > > > > > > > > > > Thanks for all the responses. I went through the emails and > > > > aggregated > > > > > the > > > > > proposals to give us an idea on where we stand at this point. > > > > > > > > > > I only included the improvements in the list and left on the side > > > the > > > > > bug > > > > > fixes. > > > > > Regarding bug fixes, I wonder if we should not have discussions > > > every > > > > > month > > > > > to discuss what are the important issues that should be fixed in > > > > > priority. > > > > > I feel that we sometimes tend to forget old issues even if they > > are > > > > > more > > > > > important than some new ones. > > > > > > > > > > Do not hesitate to tell me if I missed something or > > misinterpreted > > > > some > > > > > proposal. > > > > > > > > > > *Query side improvements:* > > > > > > > > > > * Storage Attached Index or SAI. The CEP can be found at > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-7%3A+Storage+Attached+Index > > > > > * Add support for OR predicates in the CQL where clause > > > > > * Allow to aggregate by time intervals (CASSANDRA-11871) and > > > allow > > > > > UDFs > > > > > in GROUP BY clause > > > > > * Ability to read the TTL and WRITE TIME of an element in a > > > > > collection > > > > > (CASSANDRA-8877) > > > > > * Multi-Partition LWTs > > > > > * Materialized views hardening: Addressing the different > > > > Materialized > > > > > Views issues (see CASSANDRA-15921 and [1] for some of the work > > > > > involved) > > > > > > > > > > *Security improvements:* > > > > > > > > > > * SSTables encryption (CASSANDRA-9633) > > > > > * Add support for Dynamic Data Masking (CEP pending) > > > > > * Allow the creation of roles that have the ability to assign > > > > > arbitrary > > > > > privileges, or scoped privileges without also granting those > > roles > > > > > access > > > > > to database objects. > > > > > * Filter rows from system and system_schema based on users > > > > > permissions > > > > > (CASSANDRA-15871) > > > > > > > > > > *Performance improvements:* > > > > > > > > > > * Trie-based index format (CEP pending) > > > > > * Trie-based memtables (CEP pending) > > > > > * Paxos improvements: Paxos / LWT implementation that would > > > enable > > > > > the > > > > > database to serve serial writes with two round-trips and serial > > > reads > > > > > with > > > > > one round-trip in the uncontended case > > > > > > > > > > *Safety/Usability improvements:* > > > > > > > > > > * Guardrails. The CEP can be found at > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/CASSANDRA/%28DRAFT%29+-+CEP-3%3A+Guardrails > > > > > * Add ability to track state in repair (CASSANDRA-15399) > > > > > * Repair coordinator improvements (CASSANDRA-15399) > > > > > * Make incremental backup configurable per keyspace and table > > > > > (CASSANDRA-15402) > > > > > * Add ability to blacklist a CQL partition so all requests are > > > > > ignored > > > > > (CASSANDRA-12106) > > > > > * Add default and required keyspace replication options > > > > > (CASSANDRA-14557) > > > > > * Transactional Cluster Metadata: Use of transactions to > > > propagate > > > > > cluster metadata > > > > > * Downgrade-ability: Ability to downgrade to downgrade in the > > > event > > > > > that > > > > > a serious issue has been identified > > > > > > > > > > *Pluggability improvements:* > > > > > > > > > > * Pluggable schema manager (CEP pending) > > > > > * Pluggable filesystem (CEP pending) > > > > > * Pluggable authenticator for CQLSH (CASSANDRA-16456). A CEP > > > draft > > > > > can be > > > > > found at > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1_G-OZCAEmDyuQuAN2wQUYUtZBEJpMkHWnkYELLhqvKc/edit > > > > > * Memtable API (CEP pending). The goal being to allow > > > improvements > > > > > such > > > > > as CASSANDRA-13981 to be easily plugged into Cassandra > > > > > > > > > > *Memtable pluggable implementation:* > > > > > > > > > > * Enable Cassandra for Persistent Memory (CASSANDRA-13981) > > > > > > > > > > *Other tools:* > > > > > > > > > > * CQL compatibility test suite (CEP pending) > > > > > > > > > > Le jeu. 22 avr. 2021 à 16:11, Benjamin Lerer <b.le...@gmail.com> > > a > > > > > écrit : > > > > > > > > > > > Finally, I think it's important we work to maintain trunk in a > > > > > shippable > > > > > >> state. > > > > > > > > > > > > > > > > > > I am +100 on this. Bringing Cassandra to such a state was a > > huge > > > > > effort > > > > > > and keeping it that way will help us to ensure the quality of > > the > > > > > > releases. > > > > > > > > > > > > Le jeu. 15 avr. 2021 à 17:30, Scott Andreas < > > > sc...@paradoxica.net> > > > > a > > > > > > écrit : > > > > > > > > > > > >> Thanks for starting this discussion, Benjamin! > > > > > >> > > > > > >> I share others’ enthusiasm on this thread for improvements to > > > > > secondary > > > > > >> indexes, trie-based partition indexes, guardrails, and > > > encryption > > > > > at rest. > > > > > >> > > > > > >> Here are some other post-4.0 areas for investment that have > > been > > > > on > > > > > my > > > > > >> mind: > > > > > >> > > > > > >> – Transactional Cluster Metadata > > > > > >> Migrating from optimistic modification and propagation of > > > cluster > > > > > >> metadata via gossip to a transactional implementation opens a > > > lot > > > > of > > > > > >> possibilities. Token movements and instance replacements get > > > safer > > > > > and > > > > > >> faster. Schema changes can be made atomic, enabling users to > > > > > execute DDL > > > > > >> rapidly without waiting for convergence. Operations like > > > > expansions > > > > > and > > > > > >> shrinks become easier to automate with less care and feeding. > > > > > >> > > > > > >> – Paxos improvements > > > > > >> During discussion on C-12126, Benedict expressed interest in > > > > > post-4.0 > > > > > >> improvements that can be made to Cassandra’s Paxos / LWT > > > > > implementation > > > > > >> that would enable the database to serve serial writes with two > > > > > round-trips > > > > > >> and serial reads with one round-trip in the uncontended case. > > > For > > > > > many > > > > > >> cross-WAN serial use cases, this may halve the latency of CAS > > > > > queries. > > > > > >> > > > > > >> – Multi-Partition LWTs > > > > > >> LWT is a great primitive, but modeling applications with the > > > > > constraint > > > > > >> of single-key CAS can be a game of Twister. Extending the > > paxos > > > > > >> improvements discussed above to enable multi-partition CAS > > would > > > > > enable > > > > > >> users of Apache Cassandra to perform serial operations across > > > > > partition > > > > > >> boundaries. > > > > > >> > > > > > >> – Downgrade-ability > > > > > >> I also see “downgradeability” as important to future new > > release > > > > > >> adoption. Taking file format changes as an example, it’s > > > currently > > > > > not > > > > > >> possible to downgrade in the event that a serious issue has > > been > > > > > identified > > > > > >> – unless you’re able to host-replace yourself out after > > > upgrading > > > > > one > > > > > >> replica, or revert to a pre-upgrade snapshot and accept data > > > loss. > > > > > It would > > > > > >> be excellent if it were possible for v.next to continue > > writing > > > > the > > > > > >> previous SSTable/commitlog/hint/etc. format until a switch is > > > > > flipped to > > > > > >> opt into new file formats. Apache HDFS takes a similar > > approach, > > > > > enabling > > > > > >> downgrade until NameNode metadata is finalized [1]. This would > > > be > > > > an > > > > > >> excellent capability to have in Apache Cassandra, and > > > dramatically > > > > > lower > > > > > >> the stakes for new release adoption. > > > > > >> > > > > > >> On pluggability / disaggregation: > > > > > >> I agree that these are important themes. We’ll want to bring a > > > lot > > > > > of > > > > > >> care and attention to this work. Disaggregation can open a lot > > > of > > > > > >> possibilities - with the drawback of future changes being > > > > > restricted to the > > > > > >> defined interface and an inability to optimize across > > interface > > > > > boundaries. > > > > > >> We can probably hit a sweet spot, though. > > > > > >> > > > > > >> Toolchains to validate implementations of pluggable components > > > > will > > > > > >> become very important. It would be bad for the project’s users > > > if > > > > > bundled > > > > > >> implementations were of uneven quality or supported subsets of > > > > > >> functionality. Converging on a common validation toolchain for > > > > > pluggable > > > > > >> subsystems can help us ensure that quality while minimizing > > the > > > > > effort > > > > > >> required to test new implementations. > > > > > >> > > > > > >> Finally, I think it's important we work to maintain trunk in a > > > > > shippable > > > > > >> state. This might look like major changes and new features > > > hiding > > > > > behind > > > > > >> feature flags that enable users to selectively enable them as > > > > > development > > > > > >> and validation proceeds, with new code executed regardless of > > > the > > > > > flag held > > > > > >> to a higher standard. > > > > > >> > > > > > >> Cheers, > > > > > >> > > > > > >> – Scott > > > > > >> > > > > > >> [1] > > > > > >> > > > > > > > > > > > > > > > > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html > > > > > >> > > > > > >> > > > > > >> ________________________________________ > > > > > >> From: guo Maxwell <cclive1...@gmail.com> > > > > > >> Sent: Wednesday, April 14, 2021 10:25 PM > > > > > >> To: dev@cassandra.apache.org > > > > > >> Subject: Re: [DISCUSSION] Next release roadmap > > > > > >> > > > > > >> +1 > > > > > >> > > > > > >> Brandon Williams <dri...@gmail.com> 于2021年4月15日周四 上午4:48写道： > > > > > >> > > > > > >> > Agreed. Everyone just please keep in mind this thread is > > for > > > > > roadmap > > > > > >> > contributions you plan to make, not contributions you would > > > like > > > > > to > > > > > >> > see. > > > > > >> > > > > > > >> > On Wed, Apr 14, 2021 at 3:45 PM Nate McCall < > > > zznat...@gmail.com > > > > > > > > > > wrote: > > > > > >> > > > > > > > >> > > Agree with Stefan 100% on this. We need to move towards > > > > > pluggability. > > > > > >> Our > > > > > >> > > users are asking for it, it makes sense architecturally, > > and > > > > > people > > > > > >> are > > > > > >> > > doing it anyway. > > > > > >> > > > > > > > >> > > > > > > > >> > > ... > > > > > >> > > > for me definitely > > > > > >> https://issues.apache.org/jira/browse/CASSANDRA-9633 > > > > > >> > > > > > > > > >> > > > I am surprised nobody mentioned this in the previous > > > > answers, > > > > > there > > > > > >> is > > > > > >> > > > ~50 people waiting for it to happen and multiple people > > > > > working on > > > > > >> it > > > > > >> > > > seriously and wanting that feature to be there for so so > > > > long. > > > > > >> > > > ... > > > > > >> > > > > > > >> > > > > > > --------------------------------------------------------------------- > > > > > >> > To unsubscribe, e-mail: > > dev-unsubscr...@cassandra.apache.org > > > > > >> > For additional commands, e-mail: > > > dev-h...@cassandra.apache.org > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> -- > > > > > >> you are the apple of my eye ! > > > > > >> > > > > > >> > > > > > --------------------------------------------------------------------- > > > > > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > > > >> For additional commands, e-mail: > > dev-h...@cassandra.apache.org > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
