[
https://issues.apache.org/jira/browse/IGNITE-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-4521:
------------------------------------
Component/s: (was: SQL)
cache
> Allow org.apache.ignite.IgniteCache#loadCache to use also a producer/stream
> of custom queries
> ---------------------------------------------------------------------------------------------
>
> Key: IGNITE-4521
> URL: https://issues.apache.org/jira/browse/IGNITE-4521
> Project: Ignite
> Issue Type: Improvement
> Components: cache
> Affects Versions: 1.8
> Reporter: Anghel Botos
>
> As suggested in the comments from
> https://issues.apache.org/jira/browse/IGNITE-4164, we are using
> {{org.apache.ignite.IgniteCache#loadCache}} with a large number of custom
> queries to allow for the parallel load of the cache. The custom queries are
> created by our code in a similar manner as in
> {{org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache}} when
> called with no custom queries. The problem that we face is the fact that the
> generation of these queries takes a while (on the order of minutes for our
> data set), time which could already be spent loading data into the cache,
> once the first queries are available.
> The suggestion for improvement is to change the signature of the
> {{org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache}} and
> {{org.apache.ignite.IgniteCache#loadCache}} method to use a Producer/Stream
> for the custom queries, and the workers in the thread pool doing the actual
> loading should just get queries from this stream until it is exhausted/get
> from the producer until it returns null (or some other indicator that there
> are no more queries to be produced). This would shorten the total time
> required to load the cache, as it would make it more similar in behavior to
> the part of
> {{org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore#loadCache}}
> handling the case without custom queries (where as soon as the boundaries of
> a range are retrieved from the initial {{loadCacheSelRangeQry}}, the load of
> the range is automatically submitted to the thread pool for execution)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
