Hey Ilya! In fact I was talking about pre-loading the index *partitions* not pages. But in any case, it's besides the point, just a potential enhancement.
What concerns me the most is how much time the activation takes and its apparent linear growth with the size of the on-disk data. In my case, I had only 10G of data on disk and that took 1.5 minutes from node start to the active state. Now, in production I'm looking at more than 100G of data per node (primary + backup partitions). Should I expect the activation to take 15 minutes? If this is indeed the case, then Ignite Persistence looses a lot of its initial appeal - "Instantaneous Cluster Restarts" - as touted here https://ignite.apache.org/features/durablememory.html. For comparison, on a 10Gpbs network I can transfer 100G in about 90 seconds (optimistically). Next, to your point about "maybe applying updates from the WAL", I don't think that was the case (as indicated by "Finished applying WAL changes [updatesApplied=0, time=85234ms]" in the log output). Except for the initial load, the data has never changed between multiple restarts of the cluster (as well as of individual nodes). I even tried deleting all WAL files and the WAL archive to see if it makes any difference at all. In every instance the behavior was the same: 1.5 minutes of activation. So, the question remains: are these activation times expected and the current state of the implementation is as good as it gets, or there is something (configuration-wise or in Ignite code) that can be done to improve the startup time? Thanks! Andrey ________________________________ From: ilya.kasnacheev <[email protected]> Sent: Friday, January 19, 2018 4:29 AM To: [email protected] Subject: Re: Long activation times with Ignite persistence enabled Hello! As far as my understanding goes, there's no distinction in Ignite between "data pages" and "index pages" All pages in durable memory are parts of "B+ tree" and are holding both data and references to downward branches in other pages. I'm not sure it had to load all the pages. Maybe only ones that needed applying updates from WAL? Which, in case of infrequent checkpoints, were plentiful? I expect most persistent databases would have non-trivial startup times when full with data. Regards, -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
