Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
On Fri, Oct 1, 2021 at 7:20 PM bened...@apache.org wrote: > I haven’t encountered Galera – do you have any technical papers to hand? > > Yes, but it's a whole thesis :-) https://www.inf.usi.ch/faculty/pedone/Paper/199x/These-2090-Pedone.pdf I guess parts of that were presented in conference pap

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
> If I'm reading you correctly, then Accord does / could do exactly what I was > asking for: two round trips in a single DC cluster, and one roundtrip + > SkewMax when network roundtrips are >> SkewMax. Yes, in fact it’s even better than that. Even in this setup *most* transactions will still t

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
On Fri, Oct 1, 2021 at 5:30 PM bened...@apache.org wrote: > > Typical value for SkewMax in e.g. the Spanner paper, some CockroachDB > discussions = 7 ms > > I think skew max is likely to be much lower than this, even on commodity > hardware. Bear in mind that unlike Cockroach and Spanner correctn

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
You can take a look at the Accord library, as linked in the CEP: https://github.com/belliottsmith/accord It will of course be modified extensively over time, but this is the basic shape of the API that is envisaged. You can take a look at the Maelstrom implementation for how this will be integr

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Paulo Motta
> With respect to this, in my view this kind of detail is not warranted within a CEP. Software development is an exploratory process with respect to structure, and these decisions will be made as the CEP progresses. If these need to be specified upfront, then the purpose of a CEP – seeking buy in –

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
> The current document details thoroughly the protocol but in my view lacks to > illustrate what specific API, methods, modules will become available to > developers With respect to this, in my view this kind of detail is not warranted within a CEP. Software development is an exploratory proces

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
>From the CEP: Batches (including unconditional batches) on transactional tables will receive ACID properties, and grammatically correct conditional batch operations that would be rejected for operating over multiple CQL partitions will now be supported From: Paulo Motta Date: Friday, 1 Octo

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
Hi Henrik, > While I understand they are out of scope, do you happen to have already some > idea what it would require to support secondary indexes? Yes, it is likely that the approach will be the same taken by Calvin-like systems where a “reconnaissance” round is taken within the local DC to

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Paulo Motta
Can you just answer what palpable feature will be available once this CEP lands because this is still not clear to me (and perhaps to others) from the current CEP structure. The current document details thoroughly the protocol but in my view lacks to illustrate what specific API, methods, modules w

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
I’m not, though it might seem that way. I disagree with your views about how CEP should be structured. Since the CEP process was itself codified via the CEP process, if you want to recodify how CEP work, the correct way is via the CEP process itself. The discussion is being drawn in multiple di

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Paulo Motta
> If you want to impose your views on CEP structure on others, please file a CEP with the additional restrictions and guidance you want to impose and start a discussion thread. I can then respond in detail to why I perceive this approach to be flawed, in a dedicated context. This sounds very kafka

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
I disagree with you. However, this is the wrong forum to have a meta discussion about how CEP should be structured. If you want to impose your views on CEP structure on others, please file a CEP with the additional restrictions and guidance you want to impose and start a discussion thread. I ca

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Paulo Motta
> The proposal as it stands today is exceptionally thorough, more so than any other CEP to date, or any CEP is likely to be in the near future. The protocol is thoroughly described, but in my view CEP is a forum to discuss the high level architecture and plan for adding a full end-to-end enhancem

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
On Fri, Oct 1, 2021 at 4:37 PM Henrik Ingo wrote: > A known optimization for the hot rows problem is to "hint" or manually > force clients to direct all updates to the hot row to the same node, > essentially making the system leader based. This allows the database to > start processing new update

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Henrik Ingo
Hi Benedict Since you asked, I reviewed the thread a bit and found this... *secondary indexes* >> What I would like to understand better and without guessing is, what do these transactions look like from a client/user point of view? > This is a fair question, and perhaps something I should pi

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
I am of course more than happy to continue discussing CEP-15 with respect to the proposed goals, and queries about the proposed protocol. I hope people feel free to continue raising queries. If anybody disagrees with the goals or any specific part of the proposal on substantive (rather than aest

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
I think this is getting circular and unproductive. Basic disagreements about whether the CEP specifies a feature I am inclined to leave for a vote. In my view the CEP specifies several features, both immediate ones for the user (ACID batches and multi-key LWTS) and developer-focused ones around

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread Paulo Motta
I share similar feelings as jbellis that this proposal seems to be focusing on the protocol itself but lacking the actual feature that will use the protocol which IMO a key element to discuss on a CEP. It's similar to saying: hey I want to add this Tries Serialization Protocol to Cassandra, but no

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
Actually, thinking about it again, the simple optimistic protocol would in fact guarantee system forward progress (i.e. independent of transaction formulation). From: bened...@apache.org Date: Friday, 1 October 2021 at 09:14 To: dev@cassandra.apache.org Subject: Re: [DISCUSS] CEP-15: General P

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-01 Thread bened...@apache.org
Hi Jonathan, It would be great if we could achieve a bandwidth higher than 1-2 short emails per week. It remains unclear to me what your goal is, and it would help if you could make a statement like “I want Cassandra to be able to do X” so that we can respond directly to it. I am also available