There is no intention to introduce any new versions of the format specifically for DSE. If there are any further changes to the format, they will be OSS-first. In other words this support only extends to preexisting versions of the format.
Inline row index in the data file is not something we have implemented, and it's currently not in any plans. I personally am not sure how it can be done to provide a benefit: if we place it at the end of a partition, it does not help much compared to a separate file; if we place it in front, we have to buffer the partition content, which will affect write performance. In either case it may be harder to cache. Do you have something different in mind? Regards, Branimir On Mon, Nov 21, 2022 at 3:01 PM Benedict <bened...@apache.org> wrote: > Personally very pleased to see this proposal, and I’m not opposed to > easing your migration by maintaining some light support for internal file > versions - though would prefer the support have some version limit where it > can be excised (maybe for one minor version bump?) > > One implementation question: are there any plans to support inline row > index in the big sstable format files? Is this something DSE supports, and > on the roadmap just not for initial work, or currently not envisioned? > > I would anticipate significant advantage to this for many workloads, and > no downside (except for streaming - which could be resolved fairly easily > by skipping over these sections when streaming to an old node, but since we > don’t generally stream between versions I don’t see any major issue anyway). > > > On 21 Nov 2022, at 12:43, Branimir Lambov <blam...@apache.org> wrote: > > > Hi everyone, > > We would like to put CEP-25 for discussion. > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format > > The proposal describes DSE's Big Trie-indexed SSTable format, which > replaces the primary index with on-disk tries to improve lookup performance > and index size, better handle wide partitions, and remove the need to > manage key caching and index summaries. > > We would like to discuss this proposal with you. > > One of the questions that we want to ask is whether anyone objects to > maintaining full compatibility with existing files created by DataStax > Enterprise. > > Regards, > Branimir > >
