Hi Dmitry, Yes we can use those solutions in some cases, but not always.
Replication is indeed the simplest, but sadly enough replicated caches are too much overhead. We have often a minimum of 12 nodes and all data must stay in sync 12x then. We do use it for small caches that don't need a lot of updates. We use colocation all over the place. Colocation based on affinity keys is not possible though for distinct data sets with only some very specific relationships with _some other_ dataset, well known before hand. (eg. for example -not our exact use case which is more complex- items in a shopping basket with items from product inventory, both are in different caches managed on other nodes and it is not possible to denormalize such that the shopping basket knows the amount of availble items) Regards, Sven SVEN BEAUPREZ L e a d A r c h i t e c t De Kleetlaan 5, B-1831 Diegem www.theglue.com <http://www.theglue.com/> On 17/09/2018, 10:37, "Dmitriy Setrakyan" <[email protected]> wrote: Hi Sven, I will let others comment on the materialized view suggestion, but I have another question. *As we all know, joins are a nightmare in a distributed system* Have you considered collocation or replication? If a table is replicated, then it will be present on all the nodes and all joins will be fast. If two partitioned tables are colocated based on some affinity key, then joins on that affinity key will be fast as well. Both, colocation and replication are supported by Ignite. Will any of these approaches work for you? D. On Mon, Sep 17, 2018 at 11:04 AM Sven Beauprez <[email protected]> wrote: > All, > > > > We are in a situation where we have to query data over several caches. As > we all know, joins are a nightmare in a distributed system and I know there > are other means like denormalisation, but it is not sufficient anymore in > some cases we have and we need the joins. > > > > We mainly work in an OLTP context, where queries are known in advance (ie > dev time) and inpsired by following blog of several years ago, I was > wondering if the concept of “materialized views” could make it into Apache > Ignite. > > ( > https://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/ > ) > > > > It would work as follows: > > - A query must register itself in Ignite at startup time (eg. via > configuration) or during run time (eg. API call) > - The registered query is parsed and a new “view” cache is created > which will ‘cache’ the resultset of the query (could take a while, but > intermediate status can be “warming up” and “hot” when ready) > - All caches involved in the joins are now monitored for CUD > operations and relevant data is stored in the new “view” cache so the view > gets updated in real time > - All operations must be ACID compliant > - The view is queried via a very trivial select statement > > > > Would that be feasible as a new feature? > > > > > > Regards, > > > > Sven > > > > > > > > [image: cid:[email protected]] > > > > SVEN BEAUPREZ > > > > L e a d A r c h i t e c t > > > > De Kleetlaan 5, B-1831 Diegem > > www.theglue.com >
