Thanks, now I see. This works the same way as in Ignite then. If you set up an eviction policy in Ignite the data may be evicted to swap at some point of time and if a query is executed right after that the it may swap in the data back to memory. However the indexes must always be in memory.
-- Denis On Fri, Aug 19, 2016 at 12:43 PM, Michael Stolz <mst...@pivotal.io> wrote: > There is a notion of data aging out in Geode. We call it overflow to disk. > > The idea is that as data gets old you can have the records in memory > expire, and that expiry can be to disk. That's the cold data. > > You may have built an index while you were initially loading the data, and > if your predicates only hit the indexes you will still get really fast > queries if the result sets aren't large. > > If, however, you ever resort to hitting the disk-based data for a query it > is going to have to read every record that isn't in memory from disk which > is going to be extremely slow. I personally would never use Geode that way. > > > -- > Mike Stolz > Principal Engineer, GemFire Product Manager > Mobile: 631-835-4771 > > On Fri, Aug 19, 2016 at 3:35 PM, Denis Magda <magda7...@gmail.com> wrote: > >> Hi Mike, >> >> Thanks a lot for the explanation! It makes perfect sense to me. >> >> I just thought that you were able to do something with indexes in a such >> way that there is no need to preload everything from disk into memory when >> a query is executed over cold data. >> >> Then what does "execution over cold data" mean? I'm referring to the >> following sentence from the main page: >> >> *Object Query Language allows distributed query execution on hot and cold >> data, with SQL-like capabilities, including joins.* >> >> -- >> Denis >> >> >> On Fri, Aug 19, 2016 at 12:27 PM, Michael Stolz <mst...@pivotal.io> >> wrote: >> >>> Here's the thing... >>> >>> On any In-memory data grid, if you run a query before the data has been >>> loaded into memory, it is going to cause the exact same amount of disk i/o >>> to do the query as it will take to load everything into memory. >>> >>> And the system will still have to go ahead and load everything into >>> memory anyway so you're going to end up doing all that disk i/o TWICE. >>> >>> Geode DOES have a nice feature for key based access though. We actually >>> store the keys in a separate file from the data and we can load that file >>> very quickly. Then if you go after the data for one of those keys we can >>> lazily load it from disk on demand if it hasn't yet been loaded into memory. >>> >>> The Lucene integration work that is going on in Geode might also make it >>> possible to load the indexes first and lazily load the data based on >>> queries against the indexes. >>> >>> >>> -- >>> Mike Stolz >>> Principal Engineer, GemFire Product Manager >>> Mobile: 631-835-4771 >>> >>> On Fri, Aug 19, 2016 at 2:59 PM, Denis Magda <magda7...@gmail.com> >>> wrote: >>> >>>> Hello Geode community, >>>> >>>> I've been investigating possibilities of Geode Persistence for a while >>>> and still can't get it clear whether I need to have all my data in memory >>>> if I want to execute OQL queries or OQL engine works over the persistence >>>> as well. >>>> >>>> My use case is the following. During the cluster startup I don't want >>>> to wait while all the data has been pre-loaded from the persistence to RAM >>>> and want to execute OQL queries right away. Is it feasible to implement >>>> with Geode? Please provide me with the links where I can read more about >>>> this. >>>> >>>> Regards, >>>> Denis >>>> >>> >>> >> >> >> -- >> Удачи, >> Денис Магда >> > > -- Удачи, Денис Магда
