How to remove large amount of data from cache

I want to remove the old data from cache once a day, the amout of data is abount 5 millions. My main remove processes as below: SqlFieldsQuery Sql = new SqlFieldsQuery( "select id, name from Table2 where data = "); cursor = Cache.query(S

RE: Questions on Client Reconnect and Client Cache

Can someone throw some light and help on below questions From: Lakshminarayanan, Manikandan (Cognizant) Sent: Wednesday, January 11, 2017 8:54 AM To: 'user@ignite.apache.org' Subject: Questions on Client Reconnect and Client Cache Hi, Appreciate if anyone can answers below questions 1.

Streaming Sliding Window question

Hi, I went thru the docs and videos on youtube, but it is unclear whether Ignite can do streaming sliding window calculation per Affinity key. For example, if there are data streaming in the format of (user, timestamp, action) in and I'd like calculate whether the user's last five action contains

RE: compute SQL returned data.

Yes. It works. Thanks. -邮件原件- 发件人: vkulichenko [mailto:valentin.kuliche...@gmail.com] 发送时间: 2017年1月11日 9:35 收件人: user@ignite.apache.org 主题: Re: 答复: 答复: compute SQL returned data. Shawn, map() method is execute locally on the master node, so you can do all the checks outside the task and

Re: Using object as key with BinaryMarshaller

Hi guys! Thanks, seems that it's all I need With best regards Alisher Alimov alimovalis...@gmail.com > On 10 Jan 2017, at 23:53, Denis Magda wrote: > > Hi Alisher, > > You need to fields identity resolver if want to specify a set of fields to be > used for hashCode and equals. > http://

Questions on Client Reconnect and Client Cache

Hi, Appreciate if anyone can answers below questions 1. In case if Ignite server is restarted, I need to restart the client (web applications). Is there any way client can reconnect to server on server restart. I know when server restarts it allocates a different ID and because of this

AW: AW: AW: Rediscovery after Startup

I have 2 server nodes A and B. Because of my setup only B knows the IP address of A not vice-versa. But B starts first. So while starting B tries to contact A (discovery) but does not reach it. Then after a while A becomes available. A does not know anything about B. I wish now that B by itself

Re: 答复: 答复: compute SQL returned data.

Shawn, map() method is execute locally on the master node, so you can do all the checks outside the task and then execute only if needed. Will this work? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p10016.html Sent from

答复: 答复: compute SQL returned data.

Yes. For my case, before submitting jobs, I need query data in task, so if no data, I have to submit an empty job which does nothing, this is my workaround. Is there "normal" ways for this case? Thanks Shawn -邮件原件- 发件人: vkulichenko [mailto:valentin.kuliche...@gmail.com] 发送时间: 2017年1月11日

Re: AW: AW: Rediscovery after Startup

If node can't connect to any of the addresses, and can't bind to any of the addresses, it will actually wait and will join once someone is started. I probably do not understand the problem, please explain in more detail. -Val -- View this message in context: http://apache-ignite-users.70518.x6

Re: Affinity

Hi Tejas, I don't see anything obvious in the plan. Note that when you join, you still have to scan one of the sides, so if intermediate datasets after applying conditions are still large, performance can be not very good. Joins are applied in the order they appear in the plan, so you can go throu

AW: AW: Rediscovery after Startup

So it is not possible what I ask for? In my setup there is a reason why this node knows this IP and not the other way around ...

Re: Java Service Deployment

Hi, What timestamps are you referring to? If you want to measure the latency of a particular call, surround it with timestamps and print out the duration. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Java-Service-Deployment-tp9933p10011.html Sent from t

Re: AW: Rediscovery after Startup

Lukas, New node connects to the one that is already running, not other way around. Actually, for node to start it must be able to connect to one of the addresses in the IP finder, OR be able to bind to one of these addresses. So the fact that your node started means that its own address was provi

AW: Rediscovery after Startup

Hi, yea the node is started and the only one in the topology. But It tries to discover 172.17.0.5 without luck. Now when 172.17.0.5 is available I would like to try to launch the discover-process again after the startup process. How can I achieve this? I turned on DEBUG-logging on purpose! By

Re: Rediscovery after Startup

Lukas, According to the log, this node is the first one in the topology and it is successfully started, this is correct behavior. Can you clarify what is wrong? BTW, you see all these errors only because you have DEBUG logging enabled. -Val -- View this message in context: http://apache-igni

Rediscovery after Startup

Hi, can somebody please tell me how to force an ignite node programmatically to again try to discover other nodes it did not find at the startup process. I use: final TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder(); ipFinder.setShared(true); ipFinder.setAddresses(this

Re: Using object as key with BinaryMarshaller

Hi Alisher, You need to fields identity resolver if want to specify a set of fields to be used for hashCode and equals. http://apacheignite.gridgain.org/docs/binary-marshaller#section-binary-field-identity-resolver

Re: Size of an Object

getHeapMemoryUsed gives the total amount of heap memory used. This includes all overheads introduced by Ignite, any short living objects and even garbage that is not collected yet. Monitoring it is a good idea when running test or benchmarks with all data loaded, but it will not give you an estimat

Re: Size of an Object

I am getting different numbers for the size of an object. I cached only 1 object. Using the below approach, its 107699 bytes byte[] arr = ignite.configuration().getMarshaller().marshal And using cluster get heap memory used, its 846769208 bytes igniteInstance.cluster().metrics().getHeapMemoryUse

Re: class org.apache.ignite.IgniteCheckedException: Shared memory segment has been closed

Hi, Shared memory is known to be not very stable under load, and it will not be default anymore in 2.0 (and will probably be deprecated). I would recommend to disable it, just add the following into configuration: -Val -- View this message in context: http://apache-ignit

Re: Max limit for Jobs submission to Compute task

Hi Gaurav, There is no limitation and this approach should work. The number of jobs executed in parallel will be limited by number of threads on public pools and by collision SPI [1]. [1] https://apacheignite.readme.io/docs/job-scheduling -Val -- View this message in context: http://apache-i

Re: Update value in stream transformer

I see what you're saying now. Streamed value is passed to your implementation as an argument, so you can do smth like this: stmr.receiver(new StreamTransformer() { @Override public Object process(MutableEntry e, Object... args) { Object streamed = args[0]; // Entry processor l

Re: Unable to start 2 server nodes on same machine

Can you attach your configuration? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Unable-to-start-2-server-nodes-on-same-machine-tp9987p1.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: 答复: compute SQL returned data.

Hi Shawn, This means what it says - ComputeTask.map() method returned no jobs, i.e. null or empty map. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/compute-SQL-returned-data-tp9890p.html Sent from the Apache Ignite Users mailing list archive at Nabb

Re: Unable to start 2 server nodes on same machine

Btw - how do we get ignite nodes to discover each other ? I am starting two Ignite instances - 1 per JVM with tcpDiscoveryIpFinder supplied with same IP address. However, ignite instances still don't recognize each other and are supposed to share data :( On Jan 10, 2017 18:13, "chaitanya kulkarni

Re: Unable to start 2 server nodes on same machine

If I try to start two ignite nodes(2 diff JVMs) on same IP address - without multicast - things hang there too What m I doing wrong? On Jan 10, 2017 12:59, "Vladislav Pyatkov" wrote: > Hi, > > This could be of incorrect system configuration of multicast, which lead > to hangs of discovery phase

class org.apache.ignite.IgniteCheckedException: Shared memory segment has been closed

Hi I have an issue with one of my prod server. My setup is 3 jvm using Ignite in partitioned mode. The host is Red Hat server 6.8. Brief : One of my component has thrown an IgniteCheckedException. The log extract is follows 2017-01-09 01:58:16 ERROR GridJobWorker:495 - Error sending reply for jo

ignitecheckedException thrown

Hi I have a setup of 3 jvm all talking to each other using Ignite 1.6 one of my application(jvm) has an exception thrown by the Ignite framework, which I am not able to understand. The exception trace below is from one of the process. I am using Red Hat Linux server 6.8. During this exception,

Re: Using object as key with BinaryMarshaller

Hi, Alisher! I guess, it’s expected behaviour. The documentation says, that “By default when an object is serialized into the binary format, Ignite captures it's hash code and stores it alongside with the binary object fields. This way

Max limit for Jobs submission to Compute task

Hello, I was going through various documentation and example of Compute Task to find out if there is any limit to number Jobs which can be submitted to the Compute task? My use case is, I have 200,000 Jobs which I want to submit and do the aggregation (reduce) on the result of the Job execution.

Using object as key with BinaryMarshaller

Hello! I have a little problem when try to use object as key with BinaryMarshaller. Bug occurred only with BinaryMarshaller, on JdkMarshaller all is working fine as expected. Here is example: UserObject class used as key that override hashCode and equals methods but seems that BinaryMarshalle

Re: Exception while resolving topology version.

Hi Tolga, Why is minor topology version growing so fast? Are you creating caches endless during the work? On Fri, Jan 6, 2017 at 11:23 AM, dkarachentsev wrote: > Hi Tolga, > > Please attach full logs from client and server, and your Ignite > configurations. Do you constantly create/destroy cach

Re: Unable to start 2 server nodes on same machine

Hi, This could be of incorrect system configuration of multicast, which lead to hangs of discovery phase. Could you try to check it in a latest version (1.8) without multicast? On Tue, Jan 10, 2017 at 10:13 AM, chaitanya kulkarni <9...@gmail.com > wrote: > Btw this was v 1.4.0 > > On Jan

Re: Update value in stream transformer

Hi, In general, you do not know what data has been stored in the cache and which stay in the stream (it depends on the configuration of the flushing and streamer capacity). You can access the cache directly using cache.get (key), but in this case part of the data may be in the stream. Therefore, i