Jay, > Is using async/await supported for optimistic serializable transactions (and we’re doing something wrong) > or are transactional operations currently supposed to be blocking in .NET?
Short answer - yes, please use synchronous cache APIs when working with transactions. A transaction is owned by a specific thread. Transactional API calls should be done from this thread, and `Commit` should be called from this thread. When doing `await`, we most likely end up on another thread and lose transaction context. There were plans to make transactions work across multiple threads, but the ticket looks abandoned: https://issues.apache.org/jira/browse/IGNITE-4887 On Fri, Jan 7, 2022 at 2:23 PM <jay.et...@gmx.de> wrote: > Ok, we completed the testing: > > > > Lock object cache with Optimistic Serializable transactions seems to be > the way to go (thanks again Alexei). > > > > We actually now prefer this approach over “classical” reader writer locks > for two main reasons: > > > > 1) The first reason is quite obvious: When you’re using classical reader > writer locks you’re in pessimistic world. And lock ordering sounds fairly > easy – but can actually get quite complicated if your data model ever grows > to be really complex. > > In contrast: > > Optimistic Serializable means you don’t need to worry about that but > enables you to concentrate on your business logic. That’s great. Makes life > simpler. > > > > 2) The second reason might not be as obvious at first but anyone who has > allowed their users to directly trigger commands that again trigger > exclusive locking in your datastore might have experienced that this way > you can be taken out very, very quickly. So of course you figure that you > need to do sanity-checks first but doing this non-blocking is not easy when > you have strong consistency requirements (both replicating data or taking > reader locks for checking first violates the consistency requirement). Then > you’re excited to find the option of “upgradeable reader locks” but soon > after that it turns out that in practical applications there can only be a > single upgradable lock. So this doesn’t help you a lot. Then you’re back to > square one since you are basically forced to acquire a write-lock before > you can do the consistent sanity check which makes it just too easy to > flood your servers. > > In contrast: > > Optimistic Serializable enables you to do fully consistent sanity checks > of user traffic on a massive scale – without ever taking a single lock! > Gives your servers much more room to breathe until the AI kicks in. This is > a security-related feature we are starting to absolutely love! > > > > Thanks again for all the input we received on this! > > > > One minor issue around the approach remains: > > When testing with a .NET embedded thick client (2.11.1) we only got the > behaviour we needed when strictly sticking to blocking cache method calls. > As soon as we introduced async/await hell broke loose and we got tons of > different errors and wired behaviour. Outside transactions async/await > appears to work nicely. So the question is: > > > > - Is using async/await supported for optimistic serializable > transactions (and we’re doing something wrong) or are transactional > operations currently supposed to be blocking in .NET? > > > > Thanks for any pointers. > > > > Jay > > > > > > *From:* Alexei Scherbakov <alexey.scherbak...@gmail.com> > *Sent:* Thursday, 6 January 2022 12:13 > *To:* user <user@ignite.apache.org> > *Subject:* Re: RE: Transactional Reader Writer Locks > > > > Gurmehar, > > "I need to understand when we mean LOCK, is this lock is acquired on > entire Cache or on record we are trying to update . > > Please clarify ." - Ignite uses record level locking. > > > > Jay, > > I would avoid entire cache locks due to performance reasons, but if it's > really necessary, lock object cache seems the best solution. > > > > > > > > вт, 28 дек. 2021 г. в 13:39, <jay.et...@gmx.de>: > > Hi Thomas, > > > > thanks for your feedback. > > > > We originally discarded that option since it was our understanding that > this would not work as desired in a transactional context. > > > > Maybe we were wrong. Will look into it again. Thanks! > > > > Jay > > > > > > > > *From:* don.tequ...@gmx.de <don.tequ...@gmx.de> > *Sent:* Tuesday, 28 December 2021 10:59 > *To:* user@ignite.apache.org > *Subject:* Re: RE: Transactional Reader Writer Locks > > > > Instead of creating a cache with lock objects wouldn’t it be easier to use > a semaphore for each cache where you want to achieve strong reader-Writer > consistency? > > > > https://ignite.apache.org/docs/latest/data-structures/semaphore > > > > Then every time before reading/writing you acquire the semaphore first. > > > > I guess this semaphore does essentially what you’re doing manually. > > > > Regards > > Thomas. > > > > > > > > On 28.12.21 at 10:41, jay.et...@gmx.de wrote: > > And if anyone knows a different way to achieve entire-cache-locks in > Optimistic Serializable transactions: We always appreciate the help. > > > > Jay > > > > > > *From:* Gurmehar Kalra <gurmehar.ka...@hcl.com> > *Sent:* Monday, 27 December 2021 07:31 > *To:* user@ignite.apache.org; alexey.scherbak...@gmail.com > *Subject:* RE: Transactional Reader Writer Locks > > > > Hi, > > I need to understand when we mean LOCK, is this lock is acquired on entire > Cache or on record we are trying to update . > Please clarify . > > > > Regards, > > Gurmehar Singh > > > > *From:* Alexei Scherbakov <alexey.scherbak...@gmail.com> > *Sent:* 16 December 2021 22:40 > *To:* user <user@ignite.apache.org> > *Subject:* Re: Transactional Reader Writer Locks > > > > [CAUTION: This Email is from outside the Organization. Unless you trust > the sender, Don’t click links or open attachments as it may be a Phishing > email, which can steal your Information and compromise your Computer.] > > Hi. > > > > You can try OPTIMISTIC SERIALIZABLE isolation, it might have better > throughput in contending scenarios. > > But this is not the same as RW lock, because a tx can be invalidated after > a commit if a lock conflict is detected. > > No RW lock of any kind is planned, AFAIK. > > > > вт, 7 дек. 2021 г. в 23:22, <jay.et...@gmx.de>: > > Dear all, > > > > we’re running in circles with Ignite for so long now. Can anyone please > help? All our attempts to custom-build a Reader Writer Lock (/Re-entrant > Lock) for use inside transactions have failed. > > > > Background: > > - Multi-node setup > > - Very high throughput mixed read/write cache access > > - Key-Value API using transactional caches > > - Strong consistency absolute requirement > > - Transactional context required for guarantees and fault-tolerance > > > > Using Pessimistic Repeatable-Read transactions gives strong consistency > but kills performance if there’s a large number of operations on the same > cache entry (and they tend to introduce performance penalties in > entire-cache operations and difficulties in cross-cache locking as well). > All other transactional modes somehow violate the strong consistency > requirement as we see it and were able to test so far. > > > > In other distributed environments we use reader writer locks to gain both > strong consistency and high performance with mixed workloads. In Ignite > however we’re not aware that explicit locks can be used inside > transactions: The documentation clearly states so ( > https://ignite.apache.org/docs/latest/distributed-locks > <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fignite.apache.org%2Fdocs%2Flatest%2Fdistributed-locks&data=04%7C01%7Cgurmehar.kalra%40hcl.com%7Cbb66d82317d148b6221608d9c0b6ee08%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637752714176933321%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1eNvAIsE5mgD6H0CSO6IX%2BSIw2nprWcQ1KX%2B5iZfwcc%3D&reserved=0>) > and trying to custom-build a reader writer lock for use inside transactions > we always end up concluding that this may not be achievable if there are > multiple ways to implicitly acquire but none to release locks. > > > > Are we out of luck here or > > - did we miss something? > > - are there workarounds you know of? > > - are there plans to implement transactional re-entrant locks in future > releases? > > > > Jay > > > > > > > > > -- > > > Best regards, > > Alexei Scherbakov > > ::DISCLAIMER:: > ------------------------------ > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. E-mail transmission is not > guaranteed to be secure or error-free as information could be intercepted, > corrupted, lost, destroyed, arrive late or incomplete, or may contain > viruses in transmission. The e mail and its contents (with or without > referred errors) shall therefore not attach any liability on the originator > or HCL or its affiliates. Views or opinions, if any, presented in this > email are solely those of the author and may not necessarily reflect the > views or opinions of HCL or its affiliates. Any form of reproduction, > dissemination, copying, disclosure, modification, distribution and / or > publication of this message without the prior written consent of authorized > representative of HCL is strictly prohibited. If you have received this > email in error please delete it and notify the sender immediately. Before > opening any email and/or attachments, please check them for viruses and > other defects. > ------------------------------ > > > > > -- > > > Best regards, > > Alexei Scherbakov >
