Re: How does Apache Ignite distribute???

2019-10-31 Thread yann . blazart
Or simply give an attribute with a specific value to the nodes with memory and only create caches on those nodes. Envoyé de mon iPhone > Le 31 oct. 2019 à 17:19, Stephen Darlington > a écrit : > > The direct answer to your question is: implement your own > org.apache.ignite.cache.affinity.A

Re: Re:

2019-10-31 Thread yann Blazart
Another way could be to give some attributes values to this node, then create caches only on these node, no? Le jeu. 31 oct. 2019 à 08:58, BorisBelozerov a écrit : > I want to distribute more data to some nodes because they have more memory > to > save data > Can you give me some example code fo

Re:

2019-10-27 Thread yann Blazart
Hello, of my tries, with 40Gb of input data, data is not equally partitioned, documentation say that you can have 5 to 10% of differences. I saw 30% Le dim. 27 oct. 2019 à 09:56, Andrey Dolmatov a écrit : > Did you try to implement custom affinity function to control distribution? > > > On Fri,

Re: Drop index do not release memory used ?

2019-10-26 Thread yann Blazart
Yes the data pages in reuse list are reused for new objects. If you drop table, you get back all pages used in reuse list. But it seems that it's not true for drop indexes Le sam. 26 oct. 2019 à 16:18, Andrey Dolmatov a écrit : > Ignite believe that Ignite will reuse that memory. But it is a q

Re: How to know memory used by a cache or a set

2019-06-13 Thread yann Blazart
Ok, but I can't create dynamically a data region ? Because each time I receive a new file to process, I create a cachegroup to handle it, then I clean everything. Le jeu. 13 juin 2019 à 13:28, Alex Plehanov a écrit : > Hello, > > It's a known issue [1]. Now you can get cache group size via JMX o

Re: Insert into select OOM exception on java heap

2019-05-30 Thread yann Blazart
ies, update > in batches until no unmarked entries left. > > Regards, > -- > Ilya Kasnacheev > > > чт, 30 мая 2019 г. в 19:14, yann Blazart : > >> Hmmm. Can I use limit and offset ? >> >> Doing limit 1 by example and continue while insert ount = 100

Re: Insert into select OOM exception on java heap

2019-05-30 Thread yann Blazart
lazy insert ... select, so the result set will have to > be held in heap for some time. > > Regards, > -- > Ilya Kasnacheev > > > чт, 30 мая 2019 г. в 18:36, yann Blazart : > >> Hello, we have 6 nodes configured with 3Gb heap, 30Gb offheap. >> >> We st

Insert into select OOM exception on java heap

2019-05-30 Thread yann Blazart
Hello, we have 6 nodes configured with 3Gb heap, 30Gb offheap. We store lot's of data in some partitioned tables, then we are executing some "insert into select... join..." using SqlQueryField (or SqlQueryFieldEx). With tables of 5000 000 lines, we ran in a OOM error, even with lazy set to true

Re: How to know memory used by a cache or a set

2019-05-30 Thread yann Blazart
Hello. I'm almost sure I already did that. I will check tomorrow. Thks Le lun. 27 mai 2019 à 11:09, ibelyakov a écrit : > Hi, > > Did you turn on cache metrics for your data region? > > To turn the metrics on, use one of the following approaches: > 1. Set DataRegionConfiguration.setMetricsEnab

How to know memory used by a cache or a set

2019-05-25 Thread yann Blazart
Hello ! Because I have multiple process that compute files in ignite H2 database, I really need to know how memory have been used by caches and sets, but the persistence is not activated (performance) and when I try to use the MxBean for CacheGroup, it send me back a 0. Is there any solution ??

Re: Apache ignite rewrite badly SQL ???

2019-05-25 Thread yann Blazart
Hello, I answer from another mail, but it's me :) Well, using SqlQueryFieldEx I'm able to enforceJoinOrder. Now on my Qualification env, rewriting some requests, it's good now. Thanks for your help ! Now I have to find an algorithm now to optimize my requests following sql requests ! :) Le jeu.

Re: Strange behavior with tables partitioned and replicated, and indexes on join

2019-04-18 Thread yann Blazart
Hello, I must apologize, it's my bad. Indexes weren't created Shame on me and thanks for help :) Le jeu. 18 avr. 2019 à 16:18, Maxim.Pudov a écrit : > By default Ignite thinks that data is collocated. You have to tell Ignite > that it is not. For example, by adding a parameter to JDBC con

Re: Strange behavior with tables partitioned and replicated, and indexes on join

2019-04-17 Thread yann Blazart
Sorry for short answer. Document just say that if the join condition is not pk or affinity, it broadcast request instead of unicast. In my case, i use indexed columns, but I receive exception. Regards Le mer. 17 avr. 2019 à 18:36, yann Blazart a écrit : > Hi. Thanks for answer. > &g

Re: Strange behavior with tables partitioned and replicated, and indexes on join

2019-04-17 Thread yann Blazart
Hi. Thanks for answer. But it doesn't explain my case Le mer. 17 avr. 2019 à 16:56, Maxim.Pudov a écrit : > Hi, check out this article: > > https://apacheignite-sql.readme.io/docs/distributed-joins#section-non-collocated-joins > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.c

Strange behavior with tables partitioned and replicated, and indexes on join

2019-04-16 Thread yann Blazart
Hello, I really need help. First, sorry but I can’t share code because of my boss. I have two tables, normally partitioned, but I can’t collocate data (simply because other tables are related to using other columns). On these tables I have stetted my indexes, everything is done using create tabl

Some questions about store and distributed processing

2019-01-26 Thread yann Blazart
Hello all ! I will have to use Ignite because I think it's the best solution to all my concerns, but I have a few question. I have to process very big json files (200GB), with lot of objects of different type generated from it. These objects, I will have to do multiple controls on it (with a dsl)

Insert performance

2019-01-14 Thread yann Blazart
Hello, I'm doing tests to get the best performance whily using Ignite. I've to insert a lot of data, then doing some requests. First, in my tests, I'm using objects with @SqlFields annotations, cache.putAll is faster than IgniteDataStream, is it normal ? I'm using cache.putAll to send 5000 objec

Hem, registerUserClassDescriptor

2016-01-14 Thread Yann BLAZART
Hello, me ; again... I continue to replace Hazelcast by ignite in my code. I have to use Queues, and sometime, tere some things I don't understand in the Ignite code. Well, In my case, I have multiple objects to push in Queues. Each of its have its own Queues.But this object are "primary keys" o

Stupide question about Queue ?

2016-01-14 Thread Yann BLAZART
Hello, In hazelcast we can persist Queues in Database, this help me for example when consumer are down to no have OOM exception. But I don't see this possibility in Ignite docs ? This message and any attachments (the "message") is intended solely for the intended addressees and is confidenti

RE: Ignite "bugs" ?

2016-01-14 Thread Yann BLAZART
Gotcha This is this part : TcpDiscoverySpi discoverySpi=new TcpDiscoverySpi(); TcpDiscoveryJdbcIpFinder ipFinder=new TcpDiscoveryJdbcIpFinder(); ipFinder.setDataSource(dataSource); discoverySpi.setIpFinder(ipFinder); config.setDiscoverySpi(discoveryS

RE: Ignite "bugs" ?

2016-01-14 Thread Yann BLAZART
Hello. I'm not using config file I'm using java code to configure : @Produces @ApplicationScoped public Ignite produceInstance() { LOG.info("** Prepare Ignite "); IgniteConfiguration config = new IgniteConfiguration(); config.setClientMode(false); conf

RE: Ignite "bugs" ?

2016-01-13 Thread Yann BLAZART
I will From: Alexey Kuznetsov [mailto:akuznet...@gridgain.com] Sent: mercredi 13 janvier 2016 16:42 To: user@ignite.apache.org Subject: Re: Ignite "bugs" ? Yann, Take into account that this mail list does not allow binary attachments, you should upload them on some public service (like dropbox

RE: Ignite "bugs" ?

2016-01-13 Thread Yann BLAZART
o:dma...@gridgain.com] Sent: mercredi 13 janvier 2016 16:26 To: user@ignite.apache.org Subject: Re: Ignite "bugs" ? Hi Yann, Could you provide thread dumps from all the nodes? It should help to see the place that "locket" the execution. Regards, Denis On 1/13/2016 4:23 PM, Yann BLAZA

RE: Ignite "bugs" ?

2016-01-13 Thread Yann BLAZART
e shouldn't have a workaround for badly-behaved libraries that do things like that, but it's definitely a Hibernate bug. Someone else will have to talk the locking behavior you're seeing. Are you starting Ignite more than once in parallel in that case? On Wed, Jan 13, 2016 at 8:23 AM, Y

Ignite "bugs" ?

2016-01-13 Thread Yann BLAZART
Hello everybody. I'm currently evaluation Ignite vs Hazelcast on a poc. I'm facing some issues. I'm coding some integration/unit test using the ApplicationComposer of TomEE 7.0.0. I have no problem with Hazelcast on it. * The first one is when I'm start Ignite in the applicationCompose