Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

2023-09-27 Thread Henrik Ingo
are some issues > still to be solved and much logging to be reduced. > > > > I look forward to productive discussions, > > Claude > > > > [1] > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations > > [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory > > > > > > > > > -- > http://twitter.com/tjake > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [VOTE] Release Apache Cassandra 5.0-alpha1

2023-09-05 Thread Henrik Ingo
> 9. ./tools/fqltool/src/org/apache/cassandra/fqltool/commands/Dump.java > 10. ./src/java/org/apache/cassandra/net/Crc.java > 11. https://www.apache.org/legal/resolved.html#cc-by > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] The future of CREATE INDEX

2023-05-17 Thread Henrik Ingo
accommodate index implementation selection > and prescriptive enough to force the user to make a decision (and wouldn't > change the legacy behavior of the existing CREATE INDEX). In this world, > creating a legacy 2i might look something like CREATE INDEX...USING > `legacy`. &g

Re: [DISCUSS] New data type for vector search

2023-04-28 Thread Henrik Ingo
FIER is just a Term we use to denote >>>> this semantic…. Could even be NON NULL TYPE[size] >>>> >>>> On Apr 27, 2023, at 9:00 AM, Benedict wrote: >>>> >>>> >>>> That’s a bounded ring buffer, not a fixed length array.

Re: [DISCUSS] New data type for vector search

2023-04-28 Thread Henrik Ingo
resentations. >>> >>> Whether we draw any semantic link between the frozen list and whatever >>> we do here, it is fundamentally a frozen list with a restriction on its >>> size. What we’re defining here are “statically” sized arrays, whereas a >>> frozen list is es

Re: Adding vector search to SAI with heirarchical navigable small world graph index

2023-04-25 Thread Henrik Ingo
atch, but this doesn’t work in a distributed >> system where the second vector might go to a different node than the >> first. Elastic requires specifying it up front like I propose here.2. >> Driver support3. Support for updating the vector in a row that hasn’t been >> flush

Re: [DISCUSS] Next release date

2023-04-19 Thread Henrik Ingo
atabase that > risks our desired GA date. Also, again, I don't understand how cutting a > 5.0 branch makes anything substantially easier to start testing. Perhaps > I'm the only one who thinks this. If so, I'm not going to make further > noise about it. > > On Tue, A

Re: [DISCUSS] Next release date

2023-04-18 Thread Henrik Ingo
n we are talking about merges where a single merge / rebase takes more than one day. We will have to stop merging smaller work to trunk anyway, when CEP-21 is being merged. No? henrik On Tue, Apr 18, 2023 at 3:24 AM Henrik Ingo wrote: > Trying to collect a few loose ends from across thi

Re: [DISCUSS] Next release date

2023-04-17 Thread Henrik Ingo
en he agrees with a 5.0 branch freeze. This last question is basically a form of saying I hope we aren't discussing a problem that doesn't even exist? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [EXTERNAL] Re: (CVE only) support for 3,11 beyond published EOL

2023-04-17 Thread Henrik Ingo
ow) > How do other committers feel about this? > > > I am also asking specifically for 3.11 since this release has been around > so long that it might warrant longer support than what we would offer for > 4.0. > > > > This logic can also be the other way around :-) >

Re: [DISCUSS] CEP-29 CQL NOT Operator

2023-04-13 Thread Henrik Ingo
; Aha I get what you all mean: No, I actually think both are unnecessary. But yeah, certainly this latter case is a bug? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] CEP-29 CQL NOT Operator

2023-04-12 Thread Henrik Ingo
T IN for partition keys). >> >> > On Apr 4, 2023, at 2:28 AM, Piotr Kołaczkowski >> wrote: >> > >> > Hi everyone! >> > >> > I created a new CEP for adding NOT support to the query language and >> > want to start discussion aroun

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

2023-04-06 Thread Henrik Ingo
is off-topic if you like, but I value our discuss > threads being collaborative in an open-mode. Sometimes the best idea is on > the tail end of a sequence of bad and/or unpopular ideas. > > > > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

2023-04-04 Thread Henrik Ingo
ILY in the grammar we have kept KEYSPACE for the container name >>>> for a group of tables. Nearly all traditional SQL databases use DATABASE as >>>> the container name for a group of tables so it would make sense for >>>> Cassandra to adopt this naming as

Re: [DISCUSS] cep-15-accord, cep-21-tcm, and trunk

2023-03-27 Thread Henrik Ingo
odest. > > On Fri, Mar 24, 2023, at 6:15 AM, Benedict wrote: > > > Accord doesn’t have a hard dependency on CEP-21 fwiw, it just needs > linearizable epochs. This could be achieved with a much more modest patch, > essentially avoiding almost all of the insertion points of cep

Re: [EXTERNAL] [DISCUSS] Next release date

2023-03-27 Thread Henrik Ingo
esses my > two main concerns: letting the branch/release date slip too much into the > unknown, squeezing GA QA efforts, while putting in place exceptional > waivers for CEP-21 and CEP-15. > >>> > >>> I hope that in the future we will be more willing to comm

Re: [DISCUSS] cep-15-accord, cep-21-tcm, and trunk

2023-03-22 Thread Henrik Ingo
nd instead > rebase it on cep-21-tcm until such time as the latter merges to trunk, at > which point cep-15-accord can be rebased to trunk again and then merged > when ready, nominally taking up the work of CASSANDRA-18196 > <https://issues.apache.org/jira/browse/CASSANDRA-18196> again.

Re: [DISCUSS] Next release date

2023-03-01 Thread Henrik Ingo
e CEPs that won't make it? Can > these land (with or without a waiver) during the alpha phase? > 2b. If the final components to specified CEPs are not > approved/appropriate to land during alpha, would it be better if the > project commits to a one-off half-year release later in the

Re: Downgradability

2023-02-23 Thread Henrik Ingo
we? I’m pretty sure > we haven’t talked about switching the default format? > > On 23 Feb 2023, at 12:12, Henrik Ingo wrote: > >  > On Thu, Feb 23, 2023 at 11:57 AM Benedict wrote: > >> Can somebody explain to me why this is being fought tooth and nail, when >>

Re: Downgradability

2023-02-23 Thread Henrik Ingo
ed waiting for the compatibility solution first. And possibly better testing, etc. Letting them merge would be justified by the desire to have more frequent and smaller increments of work merged into trunk... well, I'm not going to repeat everything from that discussion but the same pro's an

Re: Downgradability

2023-02-22 Thread Henrik Ingo
the cluster has to be downgraded later. henrik On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo wrote: > Just this once I'm going to be really brief :-) > > Just wanted to share for reference how Mongodb implemented > downgradeability around their 4.4 version: > https://www.mongod

Re: Downgradability

2023-02-22 Thread Henrik Ingo
aying on n-versions for 5.0 and needing >>>> a bump for a 4.1 bugfix might require porting over support for new >>>> features); >>>> - it requires separate and uncoordinated solutions to the problem and >>>> switching mechanisms for each individual change. >>>> >>>> An alternative solution is to implement/complete CASSANDRA-8110 >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8110>, which provides >>>> a method of writing sstables for a target version. During upgrades, a node >>>> could be set to produce sstables corresponding to the older version, and >>>> there is a very straightforward way to implement modifications to formats >>>> like the tickets above to conform to its requirements. >>>> >>>> What do people think should be the way forward? >>>> >>>> Regards, >>>> Branimir >>>> >>>> >>>> -- >>> you are the apple of my eye ! >>> >> -- >> http://twitter.com/tjake >> >> -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Announcement: Performance testing for Cassandra

2023-02-06 Thread Henrik Ingo
Here > is the link to the fallout-tests repository > <https://github.com/datastax/fallout-tests>. > > One thing to note is that, at the moment, it is not possible to run > performance tests on trunk but rather on specific versions of Cassandra. > However, we are currentl

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
the current 2038 limit since the 1970 Unix epoch. I > wouldn't make it a configurable value, off the top of my head it would make > for some interesting bugs and debugging sessions when nodes had different > values. Food for another ticket in any case imo. > > Regards >

Re: [DISCUSS] Merging incremental feature work

2023-02-03 Thread Henrik Ingo
ld opinion and we just haven't > discussed it and switched our general behavior yet, or if this is more > controversial, so I won't burden this email with enumerating pros and cons > of the two approaches until I get a gauge of the community's temperature. > > So - what d

Re: CASSANDRA-14227 removing the 2038 limit

2023-02-03 Thread Henrik Ingo
uardrail. > > - A new NONE overflow policy is the default but everything is backwards > compatible by keeping the previous ones in place. Think upgrade scenarios > or apps relying on the previous behavior. > > - The new limit is around year 292,471,208,677 which sounds ok given the > Sun will start collapsing in 3 to 5 billion years :-) > > - Please feel free to drop by the ticket and take a look at the PR even if > it's cursory > > Thx in advance. > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Welcome Patrick McFadin as Cassandra Committer

2023-02-02 Thread Henrik Ingo
din has accepted > the invitation to become committer today. > > Thanks a lot, Patrick, for everything you have done for this project and > its community through the years. > > Congratulations and welcome! > > The Apache Cassandra PMC members > -- Henrik Ingo c. +358 40

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
un in Circle OR Jenkins. > > Sure. And thanks. But during development, did you ever run nightly tests / all tests? I wouldn't want the night after merging to trunk to be the first time those are run. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.faceboo

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
ould validate that the implementation follows the CEP. But I wouldn't expect a debate on competing approaches at this stage. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Merging CEP-15 to trunk

2023-01-30 Thread Henrik Ingo
ik PS: My current status is that I've learned a lot about accord the past week, even stumbled into a block of code that presumably should be uncommented (now or later)... But the discussion on remaining items should have been delegated to the more relevant contributors than myself. If you

Re: Merging CEP-15 to trunk

2023-01-27 Thread Henrik Ingo
25, 2023 at 5:06 PM Henrik Ingo wrote: > Thanks Benedict > > For brevity I'll respond to your email, although indirectly this is also a > continuation of my debate with Josh: > > At least on my scorecard, one issue was raised regarding the actual code: > CASSANDRA-18193 P

Re: Merging CEP-15 to trunk

2023-01-25 Thread Henrik Ingo
bottleneck our throughput as a community to the finite availability of > individuals if they're concerned about specific topics and want to be > individually involved in specific code level changes. If folks feel like > our current processes, CI infrastructure, and committer pool ri

Re: Merging CEP-15 to trunk

2023-01-25 Thread Henrik Ingo
with the effort to release 4.1. > > One thing that I wanted to ask for is when you push to CI, you or whoever > does it, to approve all jobs. > > > Thanks Ekaterina, we will be sure to fully qualify the CI result, and I > will make sure we also run your flaky test runner on th

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
he time even for small patches when we get back > to our backlog of old patches. This doesn’t mean that the previous reviews > are dismissed or people not trusted or anything like that. > But considering the size and the importance of this work, I can really see > only benefit of a final

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
ial squashing to do. That deserves a quick check by > another. Ideally this would be one of the existing reviewers (but like any > other review step, no matter how short and trivial it is, that's still an > open process). I see others already doing this when rebasing larger patches >

Re: Merging CEP-15 to trunk

2023-01-24 Thread Henrik Ingo
w long? > > Why is there a difference whether the reviewer is a committer or a PMC member? henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax> <https://www.linkedin.com/company/datastax/> <https://github.com/datastax/>

Re: Merging CEP-15 to trunk

2023-01-20 Thread Henrik Ingo
erge into trunk deserves extra eyeballs than a merge into a feature > branch. > > We can refer to this as a "higher pre-commit gateway" or a "second pass". > Either way I believe it is a good thing. > > > -- Henrik Ingo c. +358 40 569 7354 w. www.datasta

Re: Intra-project dependencies

2023-01-20 Thread Henrik Ingo
Thanks Mick and David. I've been following this silently for a few days because we already exhausted my knowledge on the topic. But it seems your collective knowledge is uncovering a nice solution. If I summarize, I like all of this: - link to SHA, not library version - use git submodules because

Re: Intra-project dependencies

2023-01-17 Thread Henrik Ingo
ng a Jenkins build from cassandra commit 123abc, should checkout/download/use the correct versions of other modules at the time 123abc was committed. henrik -- Henrik Ingo c. +358 40 569 7354 w. www.datastax.com <https://www.facebook.com/datastax> <https://twitter.com/datastax&g

Re: Intra-project dependencies

2023-01-17 Thread Henrik Ingo
| > > | GPG Key available at > https://urldefense.com/v3/__https://keybase.io/dchenbecker__;!!PbtH5S7Ebw!elHEL4NgggbLsBElKkANZ1KhFuuOpnWrUZe8RESgdwNEkPdmWuw_JQEVFlxZbPTkq7FsCt5BAn_x5pBGaqIJig$ > and | > > | > https://urldefense.

Re: Intra-project dependencies

2023-01-16 Thread Henrik Ingo
k to a SHA. > > - our releases get more complicated (our source tarballs are the asf >> releases) >> >> How? > > - handling patches cover submodules >> >> How is this different to patches affecting multiple versions in C*? > > - switching branches, an

Re: Intra-project dependencies

2023-01-16 Thread Henrik Ingo
or dtest-api in the way it requires the "same version" of a > codebase for compatibility when running upgrade tests? As the library > itself no longer has an explicit version, what I presume you meant by > logical version. > > To begi

Re: [DISCUSS] Taking another(other(other)) stab at performance testing

2023-01-10 Thread Henrik Ingo
rk of 4.0. https://arxiv.org/abs/2301.03034 henrik On Sun, Jan 8, 2023 at 5:12 AM Henrik Ingo wrote: > Hi Josh, all > > I'm sitting at an airport, so rather than participating in the comment > threads in the doc, I will just post some high level principles I've > derive

Re: [DISCUSS] Taking another(other(other)) stab at performance testing

2023-01-07 Thread Henrik Ingo
gt; Here's where we landed after the discussions earlier this month (1st page, > estimated reading time 5 minutes): > https://docs.google.com/document/d/1X5C0dQdl6-oGRr9mXVPwAJTPjkS8lyt2Iz3hWTI4yIk/edit# > > Curious to hear what other perspectives there are out there on the topic. > > E

Re: [DISCUSS] CEP-26: Unified Compaction Strategy

2022-12-20 Thread Henrik Ingo
I noticed the CEP doesn't link to this, so it should be worth mentioning that the UCS documentation is available here: https://github.com/datastax/cassandra/blob/ds-trunk/doc/unified_compaction.md Both of the above seem to do a poor job referencing the literature we've been inspired by. I will lin

Re: [DISCUSS] CEP-21: Transactional Cluster Metadata

2022-09-05 Thread Henrik Ingo
pgraded version isn't that big of a deal? Also, it should make sense to minimize the dependencies from the previous major version (without CEP-21) to the new major version (with CEP-21). If a bug is found, it's much easier to fix code in the new major version than the old and supposedly

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-24 Thread Henrik Ingo
e that data. >>>> >>>> How hard would it be for SUPERUSERs to *not* automatically get the >>>> UNMASK permission? >>>> >>>> I'll also echo the concerns around masking primary key components. >>>> It's highly likely that certain

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-23 Thread Henrik Ingo
ion to a column and role. However, I'm not sure we > need that type of granularity instead of the simplicity of attaching the > masking to the column declaration. wdyt? > > > My gut feeling likewise is that this adds complexity but little value. > >> -- Henrik Ingo +358

Re: [DISCUSS] CEP-20: Dynamic Data Masking

2022-08-22 Thread Henrik Ingo
ze masking > by using UDFs as masking functions. > > Thanks, > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on YouTube.] <https://u

Re: [DISCUSS] Cassandra ecosystem site

2022-08-07 Thread Henrik Ingo
> *http://Cassandra.Link > <https://urldefense.com/v3/__http://cassandra.link/__;!!PbtH5S7Ebw!e2PlRJmfEuZkacYBgAqGI0rJLJeePGh6jCyohtgYzSrBH-TIOesfuqmPrfneRqQ-dJiC5v14Sj-oXRnMlcMBN00cLVV0aQ$>* > : > The best resources for Apache Cassandra > > -- Henrik Ingo +358 40 569 7354 <358

Re: Cassandra project status update 2022-07-18

2022-07-19 Thread Henrik Ingo
On Mon, Jul 18, 2022 at 11:42 PM Josh McKenzie wrote: > [CI Trends] > https://butler.cassandra.apache.org/#/ > > The last three weeks show us this delightful trend: > > 3.0: 10 -> 10 > 3.11: 23 -> 15 > 4.0: 2 -> 1 > 4.1: 8 -> 10 -> 5 > trunk: 20 -> 5 > > That's 36 total failures across all our b

Re: [Marketing] For Review: Pluggable Memtable Implementations blog

2022-07-14 Thread Henrik Ingo
yAB82X1EhqWihEb35Sevv-JDE/edit > -- > > Chris Thornett > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataStaxEng> [image: Visit us on

Re: Thanks to Nate for his service as PMC Chair

2022-07-14 Thread Henrik Ingo
of Apache Cassandra PMC members can be found on: > https://cassandra.apache.org/_/community.html > > Kind regards, > > Paulo > -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.c

[DISCUSS] Cassandra ecosystem site

2022-07-04 Thread Henrik Ingo
and maintain. If we do want to start using this, maybe an immediate need would be to setup a backup of the MySQL database that hosts the contents of the site. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [ima

Re: CEP-15 multi key transaction syntax

2022-06-06 Thread Henrik Ingo
I think > it’s fine to require a client upgrade to get multiple result sets. > > Possibly. I just wanted to share an idea for consideration. IMO the temp table idea might not be too hard to implement*, but sure the syntax does feel a bit bolted on. *) I'm maybe the wrong person to judge

Re: CEP-15 multi key transaction syntax

2022-06-06 Thread Henrik Ingo
. > > On completion, an operation would return a boolean value indicating the > operation had been applied, and a result set for each named select (but not > named update). We could also support an optional RETURN keyword, which > would allow the user to only return spec

Pluggability improvements in 4.1

2022-04-26 Thread Henrik Ingo
t; pluggable cluster membership is not merged.) CEP-16 <https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH> While client side, worth mentioning: Pluggable auth for CQLSH If there are more that I don't know about, please reply and add to the list.

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-14 Thread Henrik Ingo
On Fri, Feb 11, 2022 at 8:47 PM Caleb Rackliffe wrote: > Just finished reading the latest version of the CEP. Here are my thoughts: > > - We've already talked about OR queries, so I won't rehash that, but > tokenization support seems like it might be another one of those places > where we can cut

Re: [GSOC] Call for Mentors

2022-02-11 Thread Henrik Ingo
red to complete the task? > - What warm-up tasks can the candidate do to ramp up for the project? > > The mentor will work with potential participants to refine the high level > description into smaller subtasks at a later stage (during candidate > application period). > > Cheers,

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-07 Thread Henrik Ingo
Thanks Benjamin for reviewing and raising this. While I don't speak for the CEP authors, just some thoughts from me: On Mon, Feb 7, 2022 at 11:18 AM Benjamin Lerer wrote: > I would like to raise 2 points regarding the current CEP proposal: > > 1. There are mention of some target versions and of

Re: [GSOC] Call for Mentors

2022-02-02 Thread Henrik Ingo
y before >> that so prospective participants can start engaging with the project and >> working with mentors to refine the project before submitting an application. >> >> This year I will not be able to participate as a primary mentor but I >> would be happy to c

Re: [DISCUSS] CEP-7 Storage Attached Index

2022-02-02 Thread Henrik Ingo
gt;> >> that >> >> are >> >> only >> >> loaded >> >> at runtime based on configuration settings? (sorry >> >> for >> >> the >> >> conflation >> >> on >> >> this one, but maybe

Re: Google Summer of Code 2022 - Kick-off

2022-01-21 Thread Henrik Ingo
mentor as well as submit project ideas. > > Em dom., 16 de jan. de 2022 às 19:13, Henrik Ingo < > henrik.i...@datastax.com> escreveu: > >> Hi Paulo >> >> In my experience, it works best when the mentoring organization is clear >> about who is t

Re: Google Summer of Code 2022 - Kick-off

2022-01-16 Thread Henrik Ingo
; >> to create and tag these tasks with the "gsoc" tag. >>> >> >>> >> Please note that mentoring a GSoC project is not only a good way to >>> attract >>> >> new members to the community but also a great way of recruiting new >>> memb

Re: [DISCUSS] Releasable trunk and quality

2022-01-06 Thread Henrik Ingo
mitting first to trunk, then in descending order to each stable branch, you can just check the oldest branch to verify the chain. Not everyone followed this style, but I would always append the cherry pick message, so that the last commit (to the oldest stable branch) would contain the chain of git

Re: [DISCUSS] Periodic snapshot publishing with minor version bumps

2021-12-21 Thread Henrik Ingo
luate after our next annual release.) > > ----- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Henrik Ingo +358 40 569 7354 <3

Re: [DISCUSS] Releasable trunk and quality

2021-12-21 Thread Henrik Ingo
t; > different > > > branches that are developed separately (no merge tracking) is more > > > complicated. So yeah, I see value in those merge commits. I'm not > against > > > trying something new, just would appreciate a bit more exposure to it > > > before m

Re: Paxos repairs in CEP-14

2021-12-05 Thread Henrik Ingo
everything you've explained, did I understand correctly that (focusing on a single partition and only LWT writes...) I can in any event stream commit logs from a majority of replicas, merge them, and such a merged log must contain all committed transactions to that partition. (And this should hav

Re: Paxos repairs in CEP-14

2021-12-05 Thread Henrik Ingo
of the Paxos replication/repair process, since such a log can have other uses. Even with all of the above, I'm still left wondering: does the repair process (with the above modification, say) result in a node having all writes that happened before t, or is it only guaranteed to have the most rece

Paxos repairs in CEP-14

2021-12-04 Thread Henrik Ingo
contain a complete log? In particular, I'm trying to parse the significance of "or witnessing something newer"? (Use case for this last question could be point in time restore, aka continuous backup, or also streaming writes to a downstream system.) henrik -- Henrik Ingo +

Re: [DISCUSS] Nested YAML configs for new features

2021-11-24 Thread Henrik Ingo
nding on > > user preference (example: > > track_warnings.row_index_size.abort_threshold, using the '.' notation > > to represent nested structures) > > > > Thoughts? > > > > ----

Re: [DISCUSS] Releasable trunk and quality

2021-11-17 Thread Henrik Ingo
n this topic as a community. > > Full disclosure: running face-first into 60+ failing tests on trunk when > going through the commit process for denylisting this week brought this > topic back up for me (reminds me of when I went to merge CDC back in 3.6 > and those test failures riled

Re: [DISCUSS] Creating a new slack channel for newcomers

2021-11-12 Thread Henrik Ingo
00 > > > > > followers and that we should probably have a specific channel for > > > > > newcomers. > > > > > This proposal makes total sense to me. > > > > > > > > > > What is your opinio

Re: [DISCUSS] CEP-18: Improving Modularity

2021-10-25 Thread Henrik Ingo
to a few different subsystems. > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-18%3A+Improving+Modularity > > CASSANDRA-17044 has already been created for Schema Storage changes related > to this work and more JIRAs and PRs are to follow for the other subsystems > propos

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
server side, and it's not possible to have any additional roundtrips to the client. So the use of "interactive transactions" is supposed to distinguish from those. But you're right I may have invented the word. Historically such transactions were the norm so no additional qualifier has

Re: Tradeoffs for Cassandra transaction management

2021-10-15 Thread Henrik Ingo
tive transaction. And this is traditionally the common case, even if recent NewSQL and NoSQL databases have introduced some intriguing outside of the box thinking in this area. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/

Re: [IDEA] Read committed transaction with Accord

2021-10-14 Thread Henrik Ingo
uld not serialize access" error. > > > > It looks to me like the proposed design here would (1) not allow NOC to > > make progress at READ COMMITTED, but also (2) does not provide the tools > to > > achieve progress with SERIALIZABLE either since locking outside of t

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
able to it, the server cannot retry the transaction when concurrent > changes are found to have been applied after the reconnaissance reads (what > you call the conversational phase). > > On Tue, Oct 12, 2021 at 3:55 PM Henrik Ingo > wrote: > > > Hi all > > > &g

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
uld result in "finer grained" dependency checking and therefore cause less aborts due to occ. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https://twitter.com/DataS

Re: [IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
can be done > atomically as part of the Accord operation, and then the transaction > concurrency control arbitration mechanism kicks in. > And now I teased you to outline a different read committed transaction approach :-) henrik -- Henrik Ingo +358 40 569 7354 <358

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
the entire transaction in memory and process it? Where Cassandra is coming from, I'm not particularly alarmed by this limitation as I would expect operations on a Cassandra database to be fast and small, but it's an important limitation to call out for sure. Indeed, those who have b

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
indexes (PK, or secondary) it uses to execute itself, and nothing more. The above is saying that for a given snapshot/timestamp, the result of a statement is equally well defined by the secondary index keys used as it is by the primary keys returned from those secondary index keys. henrik -- Hen

[IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
ble semantic. For my purposes I'll just note that needing to re-execute all reads during the Accord phase (commit phase) would make the design more expensive, since the transaction is now executed twice. The goal of a simplistic light weight semantic is achieved by not doing so and claimi

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
On Tue, Oct 12, 2021 at 11:54 PM Henrik Ingo wrote: > Secondary indexes are supported without any additional work needed. > > Correction: The "transaction reads its own writes" feature would require to also store secondary index keys in the transaction state. These of course

Re: Tradeoffs for Cassandra transaction management

2021-10-12 Thread Henrik Ingo
oordinator. New interfaces: BEGIN and COMMIT. ROLLBACK. Maybe some command to declare READ COMMITTED isolation level and to get the current isolation level. Future work: A motivation for the above proposal is that the same scheme could be extended to support SNAPSHOT ISOLATION transactions. This wo

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
wy_OVs Worth calling out that we are in RDBMS land now, and the above is just a replication solution, there is no sharding anywhere. For the Serializeable paper, I struggle to even imagine how it could scale to multiple shards. For SI it's kind of easier as only write conflicts need to be ch

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
hat related state is > being updated from the same region, it might simply not be needed – at > least any time soon. > > For sure. I think we're all just trying to understand the landscape what we are talking about here, not trying to say everything should be implemented in v1

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
On Fri, Oct 1, 2021 at 4:37 PM Henrik Ingo wrote: > A known optimization for the hot rows problem is to "hint" or manually > force clients to direct all updates to the hot row to the same node, > essentially making the system leader based. This allows the database to >

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
t; Subject: Re: [DISCUSS] CEP-15: General Purpose Transactions > > No, I would expect to deliver strict serializable interactive > transactions > > using Accord. These would simply corroborate that the participating keys > > had not modified their write timestamps during the fina

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-22 Thread Henrik Ingo
still the optimal building block performance wise to do so, or whether users would then have lower consistency level but still pay the performance cost of a stricter consistency level. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.co

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-22 Thread Henrik Ingo
t;synchronized timestamp" (or Hybrid Logical Clock I guess?) scheme like in Accord is more familiar to current Cassandra semantics. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://www.datastax.com/> [image: Visit us on Twitter.] <https:/

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-21 Thread Henrik Ingo
he client could somehow use the (timestamp, id) to query a node to verify whether such a transaction was recently committed. I'm unsure whether that's convenient for a user though. henrik -- Henrik Ingo +358 40 569 7354 <358405697354> [image: Visit us online.] <https://w

Re: [DISCUSS] CEP-7 Storage Attached Index

2021-09-16 Thread Henrik Ingo
t; > > > > implementations as separate sub-modules and jar > > > > > > > > > > > > > > > > files > > > > > > > > > > > > > > > > that > > > > > > > &g

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 5:06 PM bened...@apache.org wrote: > > I was thinking that a path similar to Calvin/FaunaDB is certainly > looming in the horizon at least. > > I’m not sure which aspect of these systems you are referring to. Unless I > have misunderstood, I consider them to be strictly inf

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 12:26 PM bened...@apache.org wrote: > > whether I should just* think of this as "better and more efficient LWT” > > So, the LWT concept is a Cassandra one and doesn’t have an agreed-upon > definition. My understanding of a core feature/limitation of LWTs is that > they oper

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-07 Thread Henrik Ingo
On Tue, Sep 7, 2021 at 1:31 AM bened...@apache.org wrote: > > Of course, but we may have to be selective in our back-and-forth. We can > always take some discussion off-list to keep it manageable. > > I'll try to converge.Sorry if a few comments were a bit "editorial" in the first message. I find

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-09-06 Thread Henrik Ingo
he prototype in the project as a new source > repository, to be developed as a standalone library for integration into > Cassandra. I hope the community sees the important value proposition of > this proposal, and will adopt the CEP after this discussion, so that the > library a