Denis, Yes, if there are no concurrent updates and there are no ongoing transactions preventing the entries to be evicted, then the whole page will be purged.
2018-02-07 0:23 GMT+03:00 Denis Magda <dma...@apache.org>: > Alexey, > > Let me understand this part because it’s still not crystal clear to me: > > > Now as per the > > eviction mechanism, it is both per-page and per-entry: first, we choose a > > page which fits most for eviction, however, we cannot simply erase the > page > > because quite a lot of data structures are referring to that page. To > deal > > with it, we read keys that the chosen page contains and then clear > > individual cache entries, which in turn will clear up all necessary > links. > > > However will all the keys-values from a chosen page be removed eventually > during the eviction phase? If it’s true then it’s still a page based > eviction - we select 5 random oldest pages and purge all the key-values > from them. > > — > Denis > > > On Feb 4, 2018, at 12:05 PM, Alexey Goncharuk < > alexey.goncha...@gmail.com> wrote: > > > > Guys, > > > > To clarify the questions, I would like to reiterate over the mechanics of > > evictions and then suggest a renaming that should resolve such things in > > the future. > > > > First of all, Lucas is right - eviction policy only makes sense when > native > > persistence is disabled because what it does is actually freeing up > memory > > when a user hits the memory limit. The only way to do this is to destroy > > inserted data because there is no other way to free memory. Now as per > the > > eviction mechanism, it is both per-page and per-entry: first, we choose a > > page which fits most for eviction, however, we cannot simply erase the > page > > because quite a lot of data structures are referring to that page. To > deal > > with it, we read keys that the chosen page contains and then clear > > individual cache entries, which in turn will clear up all necessary > links. > > If there are no concurrent updates, the page becomes empty and will be > > reused for other user data. This is exactly what is explained on the wiki > > page (at least in my reading of the page). > > > > Second, at this point, I would rename page management mechanism with > > enabled persistence from 'eviction' to 'page replacement' or 'page > > swapping', because it does not destroy any data, but rather replaces a > > chosen buffer in memory from one page to another. The content of neither > > pages does not change during page replacement. This mechanism is unlikely > > to be selected by a user because the effectiveness of page replacements > > heavily depends on internal data structures and may change from version > to > > version, and may even be adaptive depending on the load pattern. > > > > Hope this resolves the confusion. > > > > --AG > > > > 2018-02-03 1:03 GMT+03:00 Denis Magda <dma...@apache.org>: > > > >> Dmitriy, > >> > >> I’m surprised to hear that Random-LRU works at the entry level when > Ignite > >> persistence is off. Frankly, this section confuses a lot: > >> https://cwiki.apache.org/confluence/display/IGNITE/ > >> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- > >> underthehood-Entryeviction <https://cwiki.apache.org/ > >> confluence/display/IGNITE/Ignite+Durable+Memory+-+under+ > >> the+hood#IgniteDurableMemory-underthehood-Entryeviction> > >> > >> While it was assumed that the entry-based eviction works only for > on-heap > >> caches: > >> https://apacheignite.readme.io/docs/evictions < > >> https://apacheignite.readme.io/docs/evictions> > >> > >> Alex G., please chime in and clarify the misunderstanding. > >> > >> — > >> Denis > >> > >>> On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> > wrote: > >>> > >>> Hi Val, > >>> > >>> I think it is quite accurate because eviction in case PDS enabled or > not > >>> has quite different purposes. > >>> > >>> 1. Let us consider PDS enabled and page eviction occurs. First of all > it > >> is > >>> page based eviction, but not entry-based. Actually data is not removed, > >> but > >>> only written to disk. We can address this page later by ID. > >>> PDS eviction is primarily the replacement of pages from memory to page > on > >>> disk. This eviction policy should ensure the maximum performance of the > >>> solution in the future. There is no data removal from grid in this > case. > >>> And Ignite does not allow users to configure the policy. If benchmarks > >> show > >>> that a change in policy results in increased performance, then we can > >>> switch policy. > >>> > >>> 2. Entry-based eviction (if there is no persistence is available) is > >>> slithly different. User will need to reload data into grid in case > entry > >> is > >>> evicted but required in cache. In that case Ignite provides selection > of > >>> policies. > >>> > >>> Sincerely, > >>> Dmirtiy Pavlov > >>> > >>> > >>> > >>> чт, 1 февр. 2018 г. в 22:24, Valentin Kulichenko < > >>> valentin.kuliche...@gmail.com>: > >>> > >>>> Guys, > >>>> > >>>> Thanks for responses. But I still fail to understand how it affects a > >> user. > >>>> > >>>> Are you saying that with persistence enabled Random-LRU is always used > >>>> regardless of what is specified in pageEvictionMode, while in > in-memory > >>>> only scenario we can also utilize Random-2-LRU for eviction? Is this > >>>> accurate? Are there any other differences? > >>>> > >>>> -Val > >>>> > >>>> On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> > >>>> wrote: > >>>> > >>>>> Hi Vyacheslav, > >>>>> > >>>>> Yes, this is right, but for now PDS page based eviction uses > >> Random-LRU, > >>>>> not Random-2-LRU. Something may be changed in future Ignite releases, > >> but > >>>>> now Random-LRU is used. I am going to research if there is some room > >> for > >>>>> improvements in this aspect (e.g. IGNITE-7507 recenly found). > >>>>> > >>>>> If clean page is evicted, nothing really happends. If dirty page is > >>>> evicted > >>>>> from region during checkpointing process run, page is stored in file > >>>> before > >>>>> eviction > >>>>> > >>>>> Some info about this can be found in > >>>>> https://cwiki.apache.org/confluence/display/IGNITE/ > >>>>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- > >>>>> underthehood-Pagebasedeviction > >>>>> > >>>>> Sincerely, > >>>>> Dmitriy Pavlov > >>>>> > >>>>> чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur <daradu...@gmail.com > >: > >>>>> > >>>>>> Hi Valentin, > >>>>>> > >>>>>> As far as I know, when persistence is enabled and Ignite can't > >>>>>> allocate new page-memory in the memory segment, then Ignite > >>>>>> automatically evict some data from RAM to PDS using Random-2-LRU > >>>>>> eviction algorithm. And there are no mechanisms to evict entries out > >>>>>> when PDS is enabled. > >>>>>> > >>>>>> I'll be glad if I am not right and someone will correct me because, > as > >>>>>> Ignite user, I can't use PDS only as RAM disk replication for the > use > >>>>>> case in my project. > >>>>>> > >>>>>> > >>>>>> On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko > >>>>>> <valentin.kuliche...@gmail.com> wrote: > >>>>>>> Folks, > >>>>>>> > >>>>>>> On "Eviction Policies" documentation page [1] we have the following > >>>>>> callout: > >>>>>>> > >>>>>>>> Configured eviction policy has no effect if Ignite persistence is > >>>>>> enabled > >>>>>>>> Note that if Ignite Persistence is enabled, then the page-based > >>>>>> evictions > >>>>>>> have no effect because the oldest pages will be purged from memory > >>>>>>> automatically. > >>>>>>> > >>>>>>> This really confuses me. Why is there a difference in how data is > >>>>> evicted > >>>>>>> from memory depending on having disk enabled or not? And how does > >>>>>> eviction > >>>>>>> work with persistence enabled then? > >>>>>>> > >>>>>>> Does anyone can clarify? > >>>>>>> > >>>>>>> [1] https://apacheignite.readme.io/docs/evictions > >>>>>>> > >>>>>>> -Val > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best Regards, Vyacheslav D. > >>>>>> > >>>>> > >>>> > >> > >> > >
