Dmitriy, Agree with Vladimir here. OPTIMISTIC + (READ_COMMITTED | REPEATABLE_READ) modes are completely unusable in real-life use-cases because they do not allow any read-write conflict detection and thus the explicit transaction statement can be omitted at all. The remaining combinations of tx concurrency and isolation can be reduced to a new enum that will match SQL isolation levels with crystal clear isolation guarantees.
I see no problem with deprecating the old API and completely removing it in 3.0. 2017-09-26 11:52 GMT+03:00 Dmitriy Setrakyan <dsetrak...@apache.org>: > On Tue, Sep 26, 2017 at 12:08 AM, Vladimir Ozerov <voze...@gridgain.com> > wrote: > > > Dima, > > > > My proposal would require two changes: > > 1) Public - for sure we will continue support old API > > > > Of course. But my assumption is that you would deprecate the old API, > right? If yes, then you will never be able to remove the deprecated API. > > 2) Internal TX engine - these changes are inevitable for transactional SQL > > support. > > > > Internally we can do any changes we like, as long as they don't change the > expected user behavior. > > > > > > So we have no freedom to choose whether to spend time on internals or > not. > > We will have to do that anyway. > > > > On Tue, Sep 26, 2017 at 9:02 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > Vladimir, > > > > > > I do not think we have a luxury of changing Ignite transaction APIs. It > > is > > > almost the same as changing Ignite put and get APIs. Too much code has > > > already been written to these APIs. The upgrade path will be so onerous > > > that no one will ever take it. > > > > > > As far as the current transaction flags, we will have to continue to > > > support them. Ideally we should extend them to Ignite SQL as well. If > > there > > > is a new transaction flag that you wish to add to Ignite SQL, please > > > suggest it here. > > > > > > D. > > > > > > On Mon, Sep 25, 2017 at 10:48 PM, Vladimir Ozerov < > voze...@gridgain.com> > > > wrote: > > > > > > > Dima, > > > > > > > > Becuase if you take any pair of our concurrency/isolation modes, you > > will > > > > not find anything similar in SQL. The main problem is that there is > no > > > > "OPTIMISTIC" or "PESSIMISTIC" in SQL world. All reads are > "optimistic", > > > but > > > > can be converted to "perssimistic" optionally, and all writes are > > > > "pessimistic". Another problem is that some of our pairs cannot be > used > > > in > > > > practice (OPTIMISTIC/READ_COMMITTED, OPTIMISTIC/REPEATABLE_READ), and > > > some > > > > duplicates each other (PESSIMISTIC/REPEATABLE_READ - > > > > PESSIMISTIC/SERIALIZABLE). This make our API very hard to explain and > > > > reason about. If you cannot explain API in 5 mins, it is broken :-) > > > > > > > > If we were many years in the past allowed to choose between Ignite > and > > > SQL > > > > approaches I would prefer SQL because it is much simpler. > > > > > > > > On Tue, Sep 26, 2017 at 1:29 AM, Dmitriy Setrakyan < > > > dsetrak...@apache.org> > > > > wrote: > > > > > > > > > Vladimir, > > > > > > > > > > This looks like a complete overhaul of our transactional behavior > > and I > > > > do > > > > > not think we are at a liberty to make such drastic changes. Can you > > > > please > > > > > explain why the current behavior would not map to the SQL > > transactions? > > > > > > > > > > D. > > > > > > > > > > On Mon, Sep 25, 2017 at 12:09 PM, Vladimir Ozerov < > > > voze...@gridgain.com> > > > > > wrote: > > > > > > > > > > > Folks, > > > > > > > > > > > > Sorry for late reply. I had a chat with several Ignite veterans > > > today. > > > > We > > > > > > tried to design transactional SQL for Ignite. One of our > questions > > > was > > > > > how > > > > > > to align SQL transactions with current Ignite transactions. And > we > > > > > failed. > > > > > > And then we came to conclusion that current transaction API is > > > > unusable. > > > > > We > > > > > > have 6 pairs of modes from API standpoint and 4 real modes. This > is > > > > very > > > > > > counterintuitive and cannot be mapped to any transactional > > framework > > > > our > > > > > > users are familiar with. > > > > > > > > > > > > So we thought how new tx API might looks like, and here is the > > draft. > > > > > > > > > > > > 1) Define new enum *TransactionIsolationLevel *(to avoid clashes > > with > > > > > > current enum) with three standard modes - READ_COMMITTED, > > > > > REPEATABLE_READ, > > > > > > SERIALIZABLE. > > > > > > 2) Define new enum *TransactionHint* - READ_ONLY, > > OPTIMISTIC_LOCKING > > > > > > 3) No more OPTIMISTIC and PESSIMISTIC. Seriously. > > > > > > 4) Reads never acuire locks > > > > > > 5) Writes always acquire locks > > > > > > 6) *IgniteCache.withReadForUpdate()* will return special facade > > > which > > > > > will > > > > > > obtain locks on reads. This is analogue of SELECT FOR UPDATE in > > SQL. > > > > > > 7) *TransactionHint.READ_ONLY* - forces transaction to throw an > > > > exception > > > > > > on any update > > > > > > 8) *TransactionHint.OPTIMISTIC_LOCKING* - turns transaction into > > our > > > > > > current deadlock-free OPTIMISTIC/SERIALIZABLE mode. Applicable > only > > > to > > > > > > SERIALIZABLE isolation level. > > > > > > 9) Define new API methods: > > > > > > - IgniteTransactions.txStart(TransactionIsolationLevel > isolation) > > > > > > - IgniteTransactions.txStart(TransactionIsolationLevel > isolation, > > > > > > TransactionHint... hints) > > > > > > 10) Deprecate old TX start methods > > > > > > > > > > > > As a result we will have simple, clean and extensible API. Which > > can > > > be > > > > > > explained to users in 5 minutes, instead of current half an hour. > > And > > > > > which > > > > > > is perfectly aligned with upcoming transactional SQL. > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > On Thu, Sep 7, 2017 at 6:48 AM, Dmitriy Setrakyan < > > > > dsetrak...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Vova, > > > > > > > > > > > > > > Thanks for doing the research. The changes you are suggesting > > are a > > > > bit > > > > > > too > > > > > > > bold, so let's discuss them in some more detail... > > > > > > > > > > > > > > On Wed, Sep 6, 2017 at 4:51 AM, Vladimir Ozerov < > > > > voze...@gridgain.com> > > > > > > > wrote: > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > We are moving towards DBMS system. None of them has a notion > of > > > > > > > > OPTIMISTIC/PESSIMISTIC transactions. Instead, they work as > > > follows: > > > > > > > > 1) Reads (SELECT) do not acquire exclusive row locks > > > > > > > > 2) Exclusive lock on read could be forced explicitly (SELECT > > ... > > > > FOR > > > > > > > > UPDATE) > > > > > > > > 3) Writes do acuire explicit row locks > > > > > > > > 4) Locks are always acquired immediately once statement is > > > executed > > > > > > > > 5) The strictest concurrency level - typically SERIALIZABLE - > > > rely > > > > on > > > > > > > > so-called *range locks* (or *predicate locks*) to track > > > > dependencies > > > > > > > > between transactions. Some vendors throw an exception in case > > of > > > > > > > conflict - > > > > > > > > these are ones where snapshot-based MVCC is used - > PostgreSQL, > > > > > Oracle. > > > > > > > > Others do aggressive locking - ones where two-phase locking > > > > algorithm > > > > > > is > > > > > > > > used - SQL Server, MySQL. > > > > > > > > > > > > > > > > As you see, there is no concept of PESSIMISTIC/OPTIMISTIC > > modes. > > > > > > Instead, > > > > > > > > all updates are "PESSIMISTIC", reads are "OPTIMISTIC" but > could > > > > > become > > > > > > > > "PESSIMISTIC" if requested explicitly, and for snapshot-based > > > > vendors > > > > > > (we > > > > > > > > are going in this direction) read-write conflicts are > resolved > > in > > > > > > manner > > > > > > > > somewhat similar to our OPTIMISTIC/SERIALIZABLE. > > > > > > > > > > > > > > > > That said, I would propose to think on how transactions could > > > look > > > > > like > > > > > > > in > > > > > > > > future Ignite versions (say, 3.0). My rough vision: > > > > > > > > > > > > > > > > 1) No OPTIMISTIC mode at all - too counterintuitive and > > complex. > > > > It's > > > > > > > only > > > > > > > > advantage is deadlock-freedom when combined with > SERIALIZABLE. > > If > > > > we > > > > > > have > > > > > > > > good deadlock detector and nice administrative capabilities, > > this > > > > > would > > > > > > > not > > > > > > > > be a problem for us. > > > > > > > > > > > > > > > > > > > > > Hm... The advantage of Optimistic Serialiazable mode is > actually > > > > > > lock-free > > > > > > > transactions. The deadlock is impossible in this case. I doubt > > any > > > > > > deadlock > > > > > > > detector would match the performance advantage we get from > > > lock-free > > > > > > > transactions. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2) Behavior of reads could be controlled through "with" > facade: > > > > > > > > V val1 = cache.get(key1); // Shared lock or > no > > > lock > > > > > > > > V val2 = cache.withForUpdate().get(key2); // Exclusive lock > > > > > > > > > > > > > > > > > > > > > > Don't like the API. We are not trying to abandon the data grid > > > > use-case > > > > > > or > > > > > > > API, we are trying to add the database use case. > > > > > > > > > > > > > > > > > > > > > > 3) REPEATABLE_READ - throw exception in case of write-write > > > > conflict > > > > > > > > > > > > > > > > > > > > > > Well, I would like to preserve the PESSIMISTIC mode. I find it > > more > > > > > > > convenient than the "withForUpdate" API. It almost seems like > you > > > are > > > > > > > trying to force the pendulum too far in the opposite direction. > > > > > > > > > > > > > > > > > > > > > > 4) SERIALIZABLE - throw exception in case of write-write and > > > > > write-read > > > > > > > > confilct (this is how our OPTIMISTIC/SERIALZABLE works now, > but > > > it > > > > > > > doesn't > > > > > > > > support predicates) > > > > > > > > > > > > > > > > > > > > > > So, no change here? Good :) > > > > > > > > > > > > > > > > > > > > > > 5) Add READ_ONLY isolation mode where updates will not be > > allowed > > > > at > > > > > > all. > > > > > > > > Such transacrtons would be able to bypass some Ignite > internals > > > to > > > > > > > achieve > > > > > > > > greater performance, what could be valuable for mostly-read > use > > > > cases > > > > > > > (e.g. > > > > > > > > OLAP). > > > > > > > > > > > > > > > > > > > > > > Love the idea. We have already seen many use cases that could > > > benefit > > > > > > from > > > > > > > it. > > > > > > > How hard is it to implement? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > Vladimir. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
