UPDATE: Looks like we're close to a solution. As Alexei pointed out, reads are concurrent in Optimistic Serializable mode. We switched to optimistic and after we found a bug in our configuration the performance with mixed workloads now is really great! Although some transactions require 100 attempts or more to successfully commit the overall system is almost 1000 times faster than with Pessimistic Repeatable_Read (at peak workload). This is awesome since now we can finally use the potential of Ignite.
What we still need right now (Gurmehar already pointed in that direction) is a way to lock entire caches. We usually do this to be 100% when it comes to ensuring consistency with inserts and deletes. Also helps to give a consistent view when calling ContainsKey or iterating over a cache. And it helps when you have transactions that change custom indexed fields. We're currently experimenting with a very simple approach based on Optimistic Serializable mode: We created a "cache lock object" for every data cache in a separate "lock cache". Then, prior to every access to a data entry we Get the corresponding cache lock object. Now if the transaction is doing a cache-wide operation (see the 5 points above) it simply updates the cache lock object as part of the transaction. The idea is that this should invalidate any other transaction that is operating on the data cache at the time, causing a performance penalty but forcing strong consistency. It's in a way very close to a reader-writer-lock. So far, this approach looks promising. Will report our findings after we complete the testing. And if anyone knows a different way to achieve entire-cache-locks in Optimistic Serializable transactions: We always appreciate the help. Jay From: Gurmehar Kalra <gurmehar.ka...@hcl.com> Sent: Monday, 27 December 2021 07:31 To: user@ignite.apache.org; alexey.scherbak...@gmail.com Subject: RE: Transactional Reader Writer Locks Hi, I need to understand when we mean LOCK, is this lock is acquired on entire Cache or on record we are trying to update . Please clarify . Regards, Gurmehar Singh From: Alexei Scherbakov <alexey.scherbak...@gmail.com <mailto:alexey.scherbak...@gmail.com> > Sent: 16 December 2021 22:40 To: user <user@ignite.apache.org <mailto:user@ignite.apache.org> > Subject: Re: Transactional Reader Writer Locks [CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.] Hi. You can try OPTIMISTIC SERIALIZABLE isolation, it might have better throughput in contending scenarios. But this is not the same as RW lock, because a tx can be invalidated after a commit if a lock conflict is detected. No RW lock of any kind is planned, AFAIK. вт, 7 дек. 2021 г. в 23:22, <jay.et...@gmx.de <mailto:jay.et...@gmx.de> >: Dear all, we're running in circles with Ignite for so long now. Can anyone please help? All our attempts to custom-build a Reader Writer Lock (/Re-entrant Lock) for use inside transactions have failed. Background: - Multi-node setup - Very high throughput mixed read/write cache access - Key-Value API using transactional caches - Strong consistency absolute requirement - Transactional context required for guarantees and fault-tolerance Using Pessimistic Repeatable-Read transactions gives strong consistency but kills performance if there's a large number of operations on the same cache entry (and they tend to introduce performance penalties in entire-cache operations and difficulties in cross-cache locking as well). All other transactional modes somehow violate the strong consistency requirement as we see it and were able to test so far. In other distributed environments we use reader writer locks to gain both strong consistency and high performance with mixed workloads. In Ignite however we're not aware that explicit locks can be used inside transactions: The documentation clearly states so (https://ignite.apache.org/docs/latest/distributed-locks <https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fignite.ap ache.org%2Fdocs%2Flatest%2Fdistributed-locks&data=04%7C01%7Cgurmehar.kalra%4 0hcl.com%7Cbb66d82317d148b6221608d9c0b6ee08%7C189de737c93a4f5a8b686f4ca99419 12%7C0%7C0%7C637752714176933321%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAi LCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1eNvAIsE5mgD6H0C SO6IX%2BSIw2nprWcQ1KX%2B5iZfwcc%3D&reserved=0> ) and trying to custom-build a reader writer lock for use inside transactions we always end up concluding that this may not be achievable if there are multiple ways to implicitly acquire but none to release locks. Are we out of luck here or - did we miss something? - are there workarounds you know of? - are there plans to implement transactional re-entrant locks in future releases? Jay -- Best regards, Alexei Scherbakov ::DISCLAIMER:: _____ The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects. _____
