Re: Why is PageSize limited to just 16kB?

ithin page. There is internal offset named > 'item' which is 2 bytes in length and has 2 bits flags in it. 2^14=16384 > > > > Sincerely, > > Dmitriy Pavlov > > > пт, 8 сент. 2017 г. в 0:22, John Wilson : > >> Hi, >> >> Ignite sets t

Java-level locks on cache entries

Hi, The code path of a cache operation, e.g. cache.put(key, value), involves locking the entry (entries) at java-level: https://github.com/apache/ignite/blob/788adc0d00e0455e06cc9acabfe9ad425fdcd65b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/atomic/Grid

Re: Java-level locks on cache entries

That one is for locking pages while a check point process is going on. Thanks, On Tue, Sep 12, 2017 at 6:57 AM, Konstantin Dudkov wrote: > Hi, > > Ignite uses page-level locks, see > https://github.com/apache/ignite/blob/43be051cd33f0e35a5bf05fa3dbe73 > 660d2dcdd2/modules/core/src/main/java/org

A quick question on WAL and Transaction

Hi, Is an Ignite transaction, *with a recovery guarantee of power loss (WALMode.DEFAULT),* considered committed only after its WAL log file has been successfully *full-sync* written to disk? If so, doesn't this incur a major slow down? Thanks, WALMode: https://github.com/apache/ignite/blob/15613

Does Ignite write READ operations of Txs to WAL?

Hi, Does Ignite write READ operations of transactions (e.g. for future auditing purposes) in the WAL? Thanks,

A quick question on Ignite's B+ tree implementation

Hi, The internal nodes of a B+ tree, by definition, store only keys while the leaf nodes store (or hold pointer to) the actual data. The documentation here, https://apacheignite.readme.io/docs/memory-architecture, states that each index node (including internal nodes) store information to access

A quick question on Cache Data Consistency

Hi, The documentation here, https://apacheignite.readme.io/docs/primary-and-backup-copies, states that regardless of write synchronization mode, cache data will always remain fully consistent across all participating nodes. Yet, the Ignite book, states that AP (of CAP theorem) is guaranteed under

Atomicity Mode and WAL

Hi, What is the purpose and difference in the use of the WAL in atomic mode vs. in transaction mode? Thanks

What is the purpose of the tag field in a lock state structure?

Hi, I'm not clear with what the 2 byte TAG field in the lock state structure is used for. Please explain. https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/util/OffheapReadWriteLock.java#L26 Thanks, John

Quick questions on replication and write synchronization mode

Hi, 1. Assume I write data item X with a FULL_ASYNC write synchronization mode, what happens if I immediately attempt to read X? Will I read an old value or do I wait till the previous writes are completed? 2. If the write mode is PRIMARY_ASYNC, will the immediate read operations on X get answered

Is WAL a memory-mapped file?

Hi, Is the WAL a memory mapped file? Is it defined per cache? Thanks.

Re: Is WAL a memory-mapped file?

apache.org/confluence/display/IGNITE/Ignite+Persistent+ > Store+-+under+the+hood#IgnitePersistentStore-underthehood-WALstructure > > Sincerely, > Dmitriy Pavlov > > ср, 1 нояб. 2017 г. в 20:36, John Wilson : > >> Hi, >> >> Is the WAL a memory mapped file? Is it defined per cache? >> >> Thanks. >> >

Re: Is WAL a memory-mapped file?

Excellent! Thanks! On Wed, Nov 1, 2017 at 12:19 PM, Alexey Kukushkin wrote: > John, > > The default mode is the slowest of the 3 WAL write modes available. The > other two are OS buffered write "LOG_ONLY" and Ignite buffered write > "BACKGROUND". > > You might need some benchmarking in your real

Quick question on Atomic Mode

Hi, I'm in atomic mode and I do a put operation on my cache and a power fail happens in the middle of the put process... does Ignite use the WAL to rollback a partial write? How does it guarantee that the atomic put is an put successfully or do-nothing operation? Thanks,

Quick Question on Check-pointing and Partition Files

Hi, Ignite documentation states that during check-pointing dirty pages will be written to partition files. I have two question based on this: 1. What exactly is a partition file? What determines the number of partitions for a Cache? 2. Are partition files immutable? or do dirty pages th

How do I register a class for a use by the BinaryMarshaller

Hi, How do I register a class, in the XML config file, to be used by the Binary Marshaller? Assume I have IgniteCache, I want the xml config equivalent for: binaryMarsh.context().descriptorForClass(Point.class, false) Thanks,

Re: How do I register a class for a use by the BinaryMarshaller

On Tue, Dec 12, 2017 at 11:25 AM, John Wilson wrote: > Hi, > > How do I register a class, in the XML config file, to be used by the > Binary Marshaller? > > Assume I have IgniteCache, I want the xml config > equivalent for: > > binaryMarsh.context().descriptorForClass(Point.class, false) > > > Thanks, >

Why does Ignite de-allocate memory regions ONLY during shutdown?

Hi, I was looking at the UnsafeMemoryProvide and it looks to me that allocated direct memory regions are deallocated only during shutdown. https://github.com/apache/ignite/blob/c5a04da7103701d4ee95910d90ba786a6ea5750b/modules/core/src/main/java/org/apache/ignite/internal/mem/unsafe/UnsafeMemoryPr

What happens if Primary Node fails during the Commit Phase

Hi, Assume the Prepare phase has completed and that the primary node has received a commit message from the coordinator. Two questions: 1. A primary node commits a transaction before it forwards a commit message to the backup nodes. True? 2. What happens if a Primary Node fails while it

Re: What happens if Primary Node fails during the Commit Phase

You're always helpful Val. Thanks! I have a question regarding Optimistic Locking 1. The documentation here, https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Key-Value+Transactions+Architecture, states that locks, for optimistic locking, are acquired during the "prepare" p

Re: What happens if Primary Node fails during the Commit Phase

I got the answer for #3 here https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Pages&links. I will post the remaining questions in a separate thread. On Mon, Feb 12, 2018 at 8:03 PM, John Wilson wrote: > You&#x

B+ Tree, Index Pages and Memory Region

I'm trying to understanding the purpose of B+ Tree and Index Pages for Apache Ignite as described here: https://apacheignite.readme.io/docs/page-memory I have a few questions: 1. What exactly does an Index Page contain? An ordered list of hash code values for keys that fall into the index

Persistence Store and sun.misc.UNSAFE

Hi, The link here, https://apacheignite.readme.io/v2.1/docs/durable-memory, states that when persistence feature is enabled all data and indexes are stored off-heap. To verify, I ran org.apache.ignite.examples.persistence.PersistenceStoreExample on Intellij and followed the code path and it looks

Re: Persistence Store and sun.misc.UNSAFE

I have both the server and client in debug mode and none of my break points were hit. Which Java classes and methods (running on the server node) do the actual off-heap store operations they get from clients? Thanks, On Wed, Aug 2, 2017 at 3:24 PM, vkulichenko wrote: > Sami, > > Streamer is run

Ignite Benchmark has very low CPU utilization, why?

Hi, I was running Yardstick-Ignite benchmark, https://github.com/apacheignite/yardstick-ignite, on a 3 node cluster. I used one node for client node and the others for server nodes. I run the IgnitePutBenchmark and found out that the CPU utilization on server nodes is very low (~20%) Can you sugg

Quick questions on Virtual Memory and Cache Memory Modes

A few quick questions: 1. Are the cache memory modes: OFFHEAP_TIERED, OFFHEAP_VALUES, and ONHEAP_TIERED, deprecated in Ignite 2.1? 2. In version 2.1, if I don't enable persistence store, do all data and indexes get stored on heap, possibly causing OME errors? 3. Is the virtual memor

Ignite Locking Mechanisms

Hi all, When a key is inserted/updated, is the data page the key-value goes into gets locked? If yes, how is this efficient? (since insertions/updates of other key-values that map to the same data page have to wait) I appreciate a detailed explanation. Thanks,

Quick questions on Evictions

Hi, I have been reading through Ignite doc and I still have these questions. I appreciate your answer. Assume my Ignite native persistence is *not *enabled: 1. if on-heap cache is also not enabled, then there are no entry-based evictions, right? 2. if on-heap cache is now enabled, does

Re: Quick questions on Evictions

I appreciate the nice explanation. I got a few more questions: 1. For the case where on-heap caching and persistent are both disabled, why does Ignite throw out out-dated pages from off-heap? Why not throw OOM error since the out-dated pages are not backed by persistent store and thro

Data Page Locking

Hi, Ignite documentation describes how and when entry-based locks are obtained, both in atomic and transactional atomicity modes. I was wondering why and when locks on data pages are required/requested -- PageMemoryImp.java shows that data pages have 8 bytes reserved for LOCK: https://github.com

Re: Data Page Locking

Thanks Mikhail! If I may ask two additional questions: 1. Is there any difference between data page eviction and check pointing (dirty pages being written to disk) when persistent store is enabled? My understanding is yes there is a difference: check pointing is a periodical process o

With onHeapCacheEnabled = false, BinaryOnHeapOutputStream is still used, why?

Hi, I'm running the CacheAPIExample below with no on-heap caching, locally using Intellij. The stack frame shows that the entry I put is written on heap (using BinaryOnheapOutputStream) and not off-heap (using BinaryOffheapOutputStream). What's going on? try (Ignite ignite = Ignition.sta

quick question on dirty pages and check pointing

Hi, Can old dirty pages purged from memory to make more room? Or only clean old pages (pages that have already been written to disk by the periodical check pointing) can be purged? If old dirty pages which has not been check-pointed can be purged, then it means the purging process (in addition to

Re: quick question on dirty pages and check pointing

rsistent Data storage mode is enabled). Only clean page may be evicted > from memory. > > In the same time too much dirty pages (for example 75%) will trigger > checkpoint process. > > Sincerely, > Dmitriy Pavlov > > чт, 7 сент. 2017 г. в 1:45, John Wilson : > >>

Why is PageSize limited to just 16kB?

Hi, Ignite sets the maximum possible size for a page to 16KB. Why? What are the drawbacks of having bigger page sizes? https://github.com/apache/ignite/blob/bd7bd226d959fbc686f6104a048106b7b944347b/modules/core/src/main/java/org/apache/ignite/configuration/MemoryConfiguration.java#L179 Thanks,