Hi Folks, Thank you for your replies! I've updated wiki according Lucas & Alexey's notes. Thank you!
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Pagereplacement(rotationwithdisk) I've named this section 'Page replacement (rotation with disk)'. I'm going to update also class naming in my current PR (3469 <https://github.com/apache/ignite/pull/3469>) Sincerely, Dmitriy Pavlov пн, 5 февр. 2018 г. в 15:25, Ivan Rakov <ivan.glu...@gmail.com>: > Guys, > > +1 for renaming "eviction" for PDS mode. > I'd choose "page replacement" over "page swapping" for two reasons: > 1) We already have "replace" metrics in DataRegionMetrics. The less we > change existing namings, the better. > 2) "Page swapping" term is common in operating systems, and that's why > we'll confuse our "page swapping" (which is essential of Ignite native > persistence) with actual OS page swapping (which usually signals about > ineffective hardware resources usage and is better to be avoided) in > many discussions later. > > Best Regards, > Ivan Rakov > > On 04.02.2018 23:42, Dmitry Pavlov wrote: > > Hi Alex, > > > > Thank you for explanation. Do you mind if I copy this explanation to wiki > > page? > > > > Strongly +1 for new term for page replacement mechanism for PDS mode. I > > like 'page swapping' because it has analogue in operating systems. > > > > New term will also identify feature by class name PageEviction... and > > PageSwap... will point to class purpose. > > > > Sincerely, > > Dmitriy Pavlov > > > > вс, 4 февр. 2018 г. в 23:05, Alexey Goncharuk < > alexey.goncha...@gmail.com>: > > > >> Guys, > >> > >> To clarify the questions, I would like to reiterate over the mechanics > of > >> evictions and then suggest a renaming that should resolve such things in > >> the future. > >> > >> First of all, Lucas is right - eviction policy only makes sense when > native > >> persistence is disabled because what it does is actually freeing up > memory > >> when a user hits the memory limit. The only way to do this is to destroy > >> inserted data because there is no other way to free memory. Now as per > the > >> eviction mechanism, it is both per-page and per-entry: first, we choose > a > >> page which fits most for eviction, however, we cannot simply erase the > page > >> because quite a lot of data structures are referring to that page. To > deal > >> with it, we read keys that the chosen page contains and then clear > >> individual cache entries, which in turn will clear up all necessary > links. > >> If there are no concurrent updates, the page becomes empty and will be > >> reused for other user data. This is exactly what is explained on the > wiki > >> page (at least in my reading of the page). > >> > >> Second, at this point, I would rename page management mechanism with > >> enabled persistence from 'eviction' to 'page replacement' or 'page > >> swapping', because it does not destroy any data, but rather replaces a > >> chosen buffer in memory from one page to another. The content of neither > >> pages does not change during page replacement. This mechanism is > unlikely > >> to be selected by a user because the effectiveness of page replacements > >> heavily depends on internal data structures and may change from version > to > >> version, and may even be adaptive depending on the load pattern. > >> > >> Hope this resolves the confusion. > >> > >> --AG > >> > >> 2018-02-03 1:03 GMT+03:00 Denis Magda <dma...@apache.org>: > >> > >>> Dmitriy, > >>> > >>> I’m surprised to hear that Random-LRU works at the entry level when > >> Ignite > >>> persistence is off. Frankly, this section confuses a lot: > >>> https://cwiki.apache.org/confluence/display/IGNITE/ > >>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- > >>> underthehood-Entryeviction <https://cwiki.apache.org/ > >>> confluence/display/IGNITE/Ignite+Durable+Memory+-+under+ > >>> the+hood#IgniteDurableMemory-underthehood-Entryeviction> > >>> > >>> While it was assumed that the entry-based eviction works only for > on-heap > >>> caches: > >>> https://apacheignite.readme.io/docs/evictions < > >>> https://apacheignite.readme.io/docs/evictions> > >>> > >>> Alex G., please chime in and clarify the misunderstanding. > >>> > >>> — > >>> Denis > >>> > >>>> On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> > >> wrote: > >>>> Hi Val, > >>>> > >>>> I think it is quite accurate because eviction in case PDS enabled or > >> not > >>>> has quite different purposes. > >>>> > >>>> 1. Let us consider PDS enabled and page eviction occurs. First of all > >> it > >>> is > >>>> page based eviction, but not entry-based. Actually data is not > removed, > >>> but > >>>> only written to disk. We can address this page later by ID. > >>>> PDS eviction is primarily the replacement of pages from memory to page > >> on > >>>> disk. This eviction policy should ensure the maximum performance of > the > >>>> solution in the future. There is no data removal from grid in this > >> case. > >>>> And Ignite does not allow users to configure the policy. If benchmarks > >>> show > >>>> that a change in policy results in increased performance, then we can > >>>> switch policy. > >>>> > >>>> 2. Entry-based eviction (if there is no persistence is available) is > >>>> slithly different. User will need to reload data into grid in case > >> entry > >>> is > >>>> evicted but required in cache. In that case Ignite provides selection > >> of > >>>> policies. > >>>> > >>>> Sincerely, > >>>> Dmirtiy Pavlov > >>>> > >>>> > >>>> > >>>> чт, 1 февр. 2018 г. в 22:24, Valentin Kulichenko < > >>>> valentin.kuliche...@gmail.com>: > >>>> > >>>>> Guys, > >>>>> > >>>>> Thanks for responses. But I still fail to understand how it affects a > >>> user. > >>>>> Are you saying that with persistence enabled Random-LRU is always > used > >>>>> regardless of what is specified in pageEvictionMode, while in > >> in-memory > >>>>> only scenario we can also utilize Random-2-LRU for eviction? Is this > >>>>> accurate? Are there any other differences? > >>>>> > >>>>> -Val > >>>>> > >>>>> On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov <dpavlov....@gmail.com > > > >>>>> wrote: > >>>>> > >>>>>> Hi Vyacheslav, > >>>>>> > >>>>>> Yes, this is right, but for now PDS page based eviction uses > >>> Random-LRU, > >>>>>> not Random-2-LRU. Something may be changed in future Ignite > releases, > >>> but > >>>>>> now Random-LRU is used. I am going to research if there is some room > >>> for > >>>>>> improvements in this aspect (e.g. IGNITE-7507 recenly found). > >>>>>> > >>>>>> If clean page is evicted, nothing really happends. If dirty page is > >>>>> evicted > >>>>>> from region during checkpointing process run, page is stored in file > >>>>> before > >>>>>> eviction > >>>>>> > >>>>>> Some info about this can be found in > >>>>>> https://cwiki.apache.org/confluence/display/IGNITE/ > >>>>>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- > >>>>>> underthehood-Pagebasedeviction > >>>>>> > >>>>>> Sincerely, > >>>>>> Dmitriy Pavlov > >>>>>> > >>>>>> чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur <daradu...@gmail.com > >>> : > >>>>>>> Hi Valentin, > >>>>>>> > >>>>>>> As far as I know, when persistence is enabled and Ignite can't > >>>>>>> allocate new page-memory in the memory segment, then Ignite > >>>>>>> automatically evict some data from RAM to PDS using Random-2-LRU > >>>>>>> eviction algorithm. And there are no mechanisms to evict entries > out > >>>>>>> when PDS is enabled. > >>>>>>> > >>>>>>> I'll be glad if I am not right and someone will correct me because, > >> as > >>>>>>> Ignite user, I can't use PDS only as RAM disk replication for the > >> use > >>>>>>> case in my project. > >>>>>>> > >>>>>>> > >>>>>>> On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko > >>>>>>> <valentin.kuliche...@gmail.com> wrote: > >>>>>>>> Folks, > >>>>>>>> > >>>>>>>> On "Eviction Policies" documentation page [1] we have the > following > >>>>>>> callout: > >>>>>>>>> Configured eviction policy has no effect if Ignite persistence is > >>>>>>> enabled > >>>>>>>>> Note that if Ignite Persistence is enabled, then the page-based > >>>>>>> evictions > >>>>>>>> have no effect because the oldest pages will be purged from memory > >>>>>>>> automatically. > >>>>>>>> > >>>>>>>> This really confuses me. Why is there a difference in how data is > >>>>>> evicted > >>>>>>>> from memory depending on having disk enabled or not? And how does > >>>>>>> eviction > >>>>>>>> work with persistence enabled then? > >>>>>>>> > >>>>>>>> Does anyone can clarify? > >>>>>>>> > >>>>>>>> [1] https://apacheignite.readme.io/docs/evictions > >>>>>>>> > >>>>>>>> -Val > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Best Regards, Vyacheslav D. > >>>>>>> > >>> > >
