Dmitriy, thank you! Guys, I've created the IEP [1] on wiki, please have a look.
[1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-34+Thin+client%3A+transactions+support чт, 28 мар. 2019 г. в 14:33, Dmitriy Pavlov <dpav...@apache.org>: > Hi, > > I've added permissions to account plehanov.alex > > Recently Infra integrated Apache LDAP with confluence, so it is possible to > login using Apache credentials. Probably we can ask infra if extra > permissions to edit pages should be added for committers. > > Sincerely, > Dmitriy Pavlov > > ср, 27 мар. 2019 г. в 13:37, Alex Plehanov <plehanov.a...@gmail.com>: > > > Vladimir, > > > > About current tx: ok, then we don't need tx() method in the interface at > > all (the same cached transaction info user can store by himself). > > > > About decoupling transactions from threads on the server side: for now, > we > > can start with thread-per-connection approach (we only can support one > > active transaction per connection, see below, so we need one additional > > dedicated thread for each connection with active transaction), and later > > change server-side internals to process client transactions in any server > > thread (not dedicated to this connection). This change will not affect > the > > thin client protocol, it only affects the server side. > > In any case, we can't support concurrent transactions per connection on > > the client side without fundamental changes to the current protocol > (cache > > operation doesn't bound to transaction or thread and the server doesn't > > know which thread on the client side do this cache operation). In my > > opinion, if a user wants to use concurrent transactions, he must use > > different connections from a connection pool. > > > > About semantics of suspend/resume on the client-side: it's absolutely > > different than server-side semantics (we don't need to do suspend/resume > to > > pass transaction between threads on the client-side), but can't be > > implemented efficiently without implemented suspend/resume on > server-side. > > > > Can anyone give me permissions to create IEP on Apache wiki? > > > > ср, 27 мар. 2019 г. в 11:59, Vladimir Ozerov <voze...@gridgain.com>: > > > > > Hi Alex, > > > > > > My comments was only about the protocol. Getting current info about > > > transaction should be handled by the client itself. It is not protocl's > > > concern. Same about other APIs and behavior in case another transaction > > is > > > attempted from the same thread. > > > > > > Putting protocol aside, transaction support is complicated matter. I > > would > > > propose to route through IEP and wide community discussion. We need to > > > review API and semantics very carefully, taking SUSPEND/RESUME in > count. > > > Also I do not see how we support client transactions efficiently > without > > > decoupling transactions from threads on the server side first. Because > > > without it you will need a dedicated server thread for every client's > > > transaction which is slow and may even crash the server. > > > > > > Vladimir. > > > > > > On Wed, Mar 27, 2019 at 11:44 AM Alex Plehanov < > plehanov.a...@gmail.com> > > > wrote: > > > > > > > Vladimir, what if we want to get current transaction info (tx() > > method)? > > > > > > > > Does close() method mapped to TX_END(rollback)? > > > > > > > > For example, this code: > > > > > > > > try(tx = txStart()) { > > > > tx.commit(); > > > > } > > > > > > > > Will produce: > > > > TX_START > > > > TX_END(commit) > > > > TX_END(rollback) > > > > > > > > Am I understand you right? > > > > > > > > About xid. There is yet another proposal. Use some unique per > > connection > > > id > > > > (integer, simple counter) for identifying the transaction on > > > > commit/rollback message. The client gets this id from the server with > > > > transaction info and sends it back to the server when trying to > > > > commit/rollback transaction. This id is not shown to users. But also > we > > > can > > > > pass from server to client real transaction id (xid) with transaction > > > info > > > > for diagnostic purposes. > > > > > > > > And one more question: what should we do if the client starts a new > > > > transaction without ending the old one? Should we end the old > > transaction > > > > implicitly (rollback) or throw an exception to the client? In my > > opinion, > > > > the first option is better. For example, if we got a previously used > > > > connection from the connection pool, we should not worry about any > > > > uncompleted transaction started by the previous user of this > > connection. > > > > > > > > ср, 27 мар. 2019 г. в 11:02, Vladimir Ozerov <voze...@gridgain.com>: > > > > > > > > > As far as SUSPEND/RESUME/SAVEPOINT - we do not support them yet, > and > > > > adding > > > > > them in future should not conflict with simple START/END > > > infrastructure. > > > > > > > > > > On Wed, Mar 27, 2019 at 11:00 AM Vladimir Ozerov < > > voze...@gridgain.com > > > > > > > > > wrote: > > > > > > > > > > > Hi Alex, > > > > > > > > > > > > I am not sure we need 5 commands. Wouldn't it be enough to have > > only > > > > two? > > > > > > > > > > > > START - accepts optional parameters, returns transaction info > > > > > > END - provides commit flag, returns void > > > > > > > > > > > > Vladimir. > > > > > > > > > > > > On Wed, Mar 27, 2019 at 8:26 AM Alex Plehanov < > > > plehanov.a...@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > >> Sergey, yes, the close is something like silent rollback. But we > > can > > > > > >> also implement this on the client side, just using rollback and > > > > ignoring > > > > > >> errors in the response. > > > > > >> > > > > > >> ср, 27 мар. 2019 г. в 00:04, Sergey Kozlov < > skoz...@gridgain.com > > >: > > > > > >> > > > > > >> > Nikolay > > > > > >> > > > > > > >> > Am I correctly understand you points: > > > > > >> > > > > > > >> > - close: rollback > > > > > >> > - commit, close: do nothing > > > > > >> > - rollback, close: do what? (I suppose nothing) > > > > > >> > > > > > > >> > Also you assume that after commit/rollback we may need to free > > > some > > > > > >> > resources on server node(s)or just do on client started TX? > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > On Tue, Mar 26, 2019 at 10:41 PM Alex Plehanov < > > > > > plehanov.a...@gmail.com > > > > > >> > > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Sergey, we have the close() method in the thick client, it's > > > > > behavior > > > > > >> is > > > > > >> > > slightly different than rollback() method (it should > rollback > > if > > > > the > > > > > >> > > transaction is not committed and do nothing if the > transaction > > > is > > > > > >> already > > > > > >> > > committed). I think we should support try-with-resource > > > semantics > > > > in > > > > > >> the > > > > > >> > > thin client and OP_TX_CLOSE will be useful here. > > > > > >> > > > > > > > >> > > Nikolay, suspend/resume didn't work yet for pessimistic > > > > > transactions. > > > > > >> > Also, > > > > > >> > > the main goal of suspend/resume operations is to support > > > > transaction > > > > > >> > > passing between threads. In the thin client, the transaction > > is > > > > > bound > > > > > >> to > > > > > >> > > the client connection, not client thread. I think passing a > > > > > >> transaction > > > > > >> > > between different client connections is not a very useful > > case. > > > > > >> > > > > > > > >> > > вт, 26 мар. 2019 г. в 22:17, Nikolay Izhikov < > > > nizhi...@apache.org > > > > >: > > > > > >> > > > > > > > >> > > > Hello, Alex. > > > > > >> > > > > > > > > >> > > > We also have suspend and resume operations. > > > > > >> > > > I think we should support them > > > > > >> > > > > > > > > >> > > > вт, 26 марта 2019 г., 22:07 Sergey Kozlov < > > > skoz...@gridgain.com > > > > >: > > > > > >> > > > > > > > > >> > > > > Hi > > > > > >> > > > > > > > > > >> > > > > Looks like I missed something but why we need > OP_TX_CLOSE > > > > > >> operation? > > > > > >> > > > > > > > > > >> > > > > Also I suggest to reserve a code for SAVEPOINT operation > > > which > > > > > >> very > > > > > >> > > > useful > > > > > >> > > > > to understand where transaction has been rolled back > > > > > >> > > > > > > > > > >> > > > > On Tue, Mar 26, 2019 at 6:07 PM Alex Plehanov < > > > > > >> > plehanov.a...@gmail.com > > > > > >> > > > > > > > > >> > > > > wrote: > > > > > >> > > > > > > > > > >> > > > > > Hello Igniters! > > > > > >> > > > > > > > > > > >> > > > > > I want to pick up the ticket IGNITE-7369 and add > > > > transactions > > > > > >> > support > > > > > >> > > > to > > > > > >> > > > > > our thin client implementation. > > > > > >> > > > > > I've looked at our current implementation and have > some > > > > > >> proposals > > > > > >> > to > > > > > >> > > > > > support transactions: > > > > > >> > > > > > > > > > > >> > > > > > Add new operations to thin client protocol: > > > > > >> > > > > > > > > > > >> > > > > > OP_TX_GET, 4000, Get current transaction for > client > > > > > >> connection > > > > > >> > > > > > OP_TX_START, 4001, Start a new transaction > > > > > >> > > > > > OP_TX_COMMIT, 4002, Commit transaction > > > > > >> > > > > > OP_TX_ROLLBACK, 4003, Rollback transaction > > > > > >> > > > > > OP_TX_CLOSE, 4004, Close transaction > > > > > >> > > > > > > > > > > >> > > > > > From the client side (java) new interfaces will be > > added: > > > > > >> > > > > > > > > > > >> > > > > > public interface ClientTransactions { > > > > > >> > > > > > public ClientTransaction txStart(); > > > > > >> > > > > > public ClientTransaction > > > txStart(TransactionConcurrency > > > > > >> > > > concurrency, > > > > > >> > > > > > TransactionIsolation isolation); > > > > > >> > > > > > public ClientTransaction > > > txStart(TransactionConcurrency > > > > > >> > > > concurrency, > > > > > >> > > > > > TransactionIsolation isolation, long timeout, int > > txSize); > > > > > >> > > > > > public ClientTransaction tx(); // Get current > > > connection > > > > > >> > > > transaction > > > > > >> > > > > > public ClientTransactions withLabel(String lb); > > > > > >> > > > > > } > > > > > >> > > > > > > > > > > >> > > > > > public interface ClientTransaction extends > > AutoCloseable { > > > > > >> > > > > > public IgniteUuid xid(); // Do we need it? > > > > > >> > > > > > public TransactionIsolation isolation(); > > > > > >> > > > > > public TransactionConcurrency concurrency(); > > > > > >> > > > > > public long timeout(); > > > > > >> > > > > > public String label(); > > > > > >> > > > > > > > > > > >> > > > > > public void commit(); > > > > > >> > > > > > public void rollback(); > > > > > >> > > > > > public void close(); > > > > > >> > > > > > } > > > > > >> > > > > > > > > > > >> > > > > > From the server side, I think as a first step (while > > > > > >> transactions > > > > > >> > > > > > suspend/resume is not fully implemented) we can use > the > > > same > > > > > >> > approach > > > > > >> > > > as > > > > > >> > > > > > for JDBC: add a new worker to each > ClientRequestHandler > > > and > > > > > >> process > > > > > >> > > > > > requests by this worker if the transaction is started > > > > > >> explicitly. > > > > > >> > > > > > ClientRequestHandler is bound to client connection, so > > > there > > > > > >> will > > > > > >> > be > > > > > >> > > > 1:1 > > > > > >> > > > > > relation between client connection and thread, which > > > process > > > > > >> > > operations > > > > > >> > > > > in > > > > > >> > > > > > a transaction. > > > > > >> > > > > > > > > > > >> > > > > > Also, there is a couple of issues I want to discuss: > > > > > >> > > > > > > > > > > >> > > > > > We have overloaded method txStart with a different set > > of > > > > > >> > arguments. > > > > > >> > > > Some > > > > > >> > > > > > of the arguments may be missing. To pass arguments > with > > > > > >> OP_TX_START > > > > > >> > > > > > operation we have the next options: > > > > > >> > > > > > * Serialize full set of arguments and use some value > > for > > > > > >> missing > > > > > >> > > > > > arguments. For example -1 for int/long types and null > > for > > > > > string > > > > > >> > > type. > > > > > >> > > > We > > > > > >> > > > > > can't use 0 for int/long types since 0 it's a valid > > value > > > > for > > > > > >> > > > > concurrency, > > > > > >> > > > > > isolation and timeout arguments. > > > > > >> > > > > > * Serialize arguments as a collection of > property-value > > > > pairs > > > > > >> > (like > > > > > >> > > > it's > > > > > >> > > > > > implemented now for CacheConfiguration). In this case > > only > > > > > >> > explicitly > > > > > >> > > > > > provided arguments will be serialized. > > > > > >> > > > > > Which way is better? The simplest solution is to use > the > > > > first > > > > > >> > option > > > > > >> > > > > and I > > > > > >> > > > > > want to use it if there were no objections. > > > > > >> > > > > > > > > > > >> > > > > > Do we need transaction id (xid) on the client side? > > > > > >> > > > > > If yes, we can pass xid along with OP_TX_COMMIT, > > > > > OP_TX_ROLLBACK, > > > > > >> > > > > > OP_TX_CLOSE operations back to the server and do > > > additional > > > > > >> check > > > > > >> > on > > > > > >> > > > the > > > > > >> > > > > > server side (current transaction id for connection == > > > > > >> transaction > > > > > >> > id > > > > > >> > > > > passed > > > > > >> > > > > > from client side). This, perhaps, will protect clients > > > > against > > > > > >> some > > > > > >> > > > > errors > > > > > >> > > > > > (for example when client try to commit outdated > > > > transaction). > > > > > >> But > > > > > >> > > > > > currently, we don't have data type IgniteUuid in thin > > > client > > > > > >> > > protocol. > > > > > >> > > > Do > > > > > >> > > > > > we need to add it too? > > > > > >> > > > > > Also, we can pass xid as a string just to inform the > > > client > > > > > and > > > > > >> do > > > > > >> > > not > > > > > >> > > > > pass > > > > > >> > > > > > it back to the server with commit/rollback operation. > > > > > >> > > > > > Or not to pass xid at all (.NET thick client works > this > > > way > > > > as > > > > > >> far > > > > > >> > > as I > > > > > >> > > > > > know). > > > > > >> > > > > > > > > > > >> > > > > > What do you think? > > > > > >> > > > > > > > > > > >> > > > > > ср, 7 мар. 2018 г. в 16:22, Vladimir Ozerov < > > > > > >> voze...@gridgain.com > > > > > >> > >: > > > > > >> > > > > > > > > > > >> > > > > > > We already have transactions support in JDBC driver > in > > > TX > > > > > SQL > > > > > >> > > branch > > > > > >> > > > > > > (ignite-4191). Currently it is implemented through > > > > separate > > > > > >> > thread, > > > > > >> > > > > which > > > > > >> > > > > > > is not that efficient. Ideally we need to finish > > > > decoupling > > > > > >> > > > > transactions > > > > > >> > > > > > > from threads. But alternatively we can change the > > logic > > > on > > > > > >> how we > > > > > >> > > > > assign > > > > > >> > > > > > > thread ID to specific transaction and "impersonate" > > thin > > > > > >> client > > > > > >> > > > worker > > > > > >> > > > > > > threads when serving requests from multiple users. > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > On Tue, Mar 6, 2018 at 10:01 PM, Denis Magda < > > > > > >> dma...@apache.org> > > > > > >> > > > > wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > > Here is an original discussion with a reference to > > the > > > > > JIRA > > > > > >> > > ticket: > > > > > >> > > > > > > > http://apache-ignite-developers.2346864.n4.nabble > . > > > > > >> > > > > > > > > > > > > com/Re-Transaction-operations-using-the-Ignite-Thin-Client- > > > > > >> > > > > > > > Protocol-td25914.html > > > > > >> > > > > > > > > > > > > >> > > > > > > > -- > > > > > >> > > > > > > > Denis > > > > > >> > > > > > > > > > > > > >> > > > > > > > On Tue, Mar 6, 2018 at 9:18 AM, Dmitriy Setrakyan > < > > > > > >> > > > > > dsetrak...@apache.org > > > > > >> > > > > > > > > > > > > >> > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Hi Dmitriy. I don't think we have a design > > proposal > > > > for > > > > > >> > > > transaction > > > > > >> > > > > > > > support > > > > > >> > > > > > > > > in thin clients. Do you mind taking this > > initiative > > > > and > > > > > >> > > creating > > > > > >> > > > an > > > > > >> > > > > > IEP > > > > > >> > > > > > > > on > > > > > >> > > > > > > > > Wiki? > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > D. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Tue, Mar 6, 2018 at 8:46 AM, Dmitriy > > Govorukhin < > > > > > >> > > > > > > > > dmitriy.govoruk...@gmail.com> wrote: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Hi, Igniters. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > I've seen a lot of discussions about thin > client > > > and > > > > > >> binary > > > > > >> > > > > > protocol, > > > > > >> > > > > > > > > but I > > > > > >> > > > > > > > > > did not hear anything about transactions > > support. > > > Do > > > > > we > > > > > >> > have > > > > > >> > > > some > > > > > >> > > > > > > draft > > > > > >> > > > > > > > > for > > > > > >> > > > > > > > > > this purpose? > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > As I understand we have several problems: > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > - thread and transaction have hard related > > (we > > > > use > > > > > >> > > > > thread-local > > > > > >> > > > > > > > > variable > > > > > >> > > > > > > > > > and thread name) > > > > > >> > > > > > > > > > - we can process only one transaction at > the > > > same > > > > > >> time > > > > > >> > in > > > > > >> > > > one > > > > > >> > > > > > > thread > > > > > >> > > > > > > > > (it > > > > > >> > > > > > > > > > mean we need hold thread per client. If > > connect > > > > 100 > > > > > >> thin > > > > > >> > > > > clients > > > > > >> > > > > > > to > > > > > >> > > > > > > > 1 > > > > > >> > > > > > > > > > server node, then need to hold 100 thread > on > > > the > > > > > >> server > > > > > >> > > > side) > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Let's discuss how we can implement > transactions > > > for > > > > > the > > > > > >> > thin > > > > > >> > > > > > client. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > -- > > > > > >> > > > > Sergey Kozlov > > > > > >> > > > > GridGain Systems > > > > > >> > > > > www.gridgain.com > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > -- > > > > > >> > Sergey Kozlov > > > > > >> > GridGain Systems > > > > > >> > www.gridgain.com > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
