Re: 回复: Ignite used memory 7 times greater than imported data

Here *Next, remove this setting giving a chance to Ignite pre-allocating the sizeit will need in practice* I meant removing of "startSize" parameter. -- Denis On Thu, Jul 14, 2016 at 7:20 AM, Denis Magda wrote: > Hi, > > There is a capacity planning guide [1] that lists how the memory is > c

Re: Does ignite support UPDATE/DELETE sql

We are currently working on adding insert/update/delete commands to Ignite. Here is the ticket you can follow: https://issues.apache.org/jira/browse/IGNITE-2294 Thanks, D. On Thu, Jul 14, 2016 at 6:58 AM, Denis Magda wrote: > Hi, > > Please properly subscribe to the Ignite's user list. Refer t

Re: How does AffinityKey mapped?

Hi val I come cross a problem while using inner join between two partition caches. The inner join result is incomplete. Following is my code. I use DTermKey as two caches' key. The inner join key is DTerm, So I use [AffinityKeyMapped] on DTermKey.DTerm. Is there any thing wrong? var offerTermCac

Re: Cassandra Cache Store Example

Ok Denis! I've some doubts in the Cassandra Persistent store. 1. In Ignite partitioned cache, number of partitions can be configured using AffinityFunction. How to configure the same for the Cassandra table ? 2. Will the partitioner used by Cassandra (MurMur3) and the Ignite returns th

Re: Understanding data store and partitioning

Hi Kamal, All the data that has been already pre-loaded by the time the new node joined and has to be located on the new node according to the new topology version will be rebalanced there [1]. However if the pre-loading is still in progress then you need to call localLoadCache() on this node bec

Re: Understanding data store and partitioning

Val, When a new node joins the cluster, should I have to call loadCache() or localLoadCache() method ? >From docs, 1. loadCache() - executes localLoadCache() on all the nodes 2. localLoadCache() - will trigger data loading only in the local node. --Kamal On Thu, Jul 14, 2016 at 4:12 AM, vkulic

Re: Writebehind retry logic

Hi Sparkle, Please properly subscribe to the mailing list so that the community can receive email notifications for your messages. Here is the instruction: http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1 sparkle_j wrote > Hi Val, > > If write behind fa

Re: Performance in case of 1 server node

Hi, See my answers inline daniels wrote > Hi Denis, > I will remove startSize. > Is there any need to add or remove anything? Nothing else at the moment. daniels wrote > Before,I used JCache(JSR 107). > Now instead of it I use Ignite(one node). Ignite is a complaint JCache implementation. H

Re: 回复: Ignite used memory 7 times greater than imported data

Hi, There is a capacity planning guide [1] that lists how the memory is consumed by Ignite and how you can control it. The first thing is that if you have a single backup then the amount of is doubled. Next, remove this setting giving a chance to Ignite pre-allocating the size it will need in pr

Re: Does ignite support UPDATE/DELETE sql

Hi, Please properly subscribe to the Ignite's user list. Refer to this page for details - https://ignite.apache.org/community/resources.html#ask See my answers inline. zhaojun08 wrote > HI ALL, > > I am new to ignite, and I have a few questions to confirm. > > 1. I want to use ignite to store

Re: How does AffinityKey mapped?

Correct! -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-does-AffinityKey-mapped-tp6260p6291.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Does ignite support UPDATE/DELETE sql

HI ALL, I am new to ignite, and I have a few questions to confirm. 1. I want to use ignite to store RDBMS in MEM. Table in RDBMS have many rows, does ignite store every row as a Java object, like the "Person" object in docs? And is it the only way to store a row in Ignite? 2. I notice that "Pers

Re: Does the HTTP REST API support the binary value?

Hi Jason, In this thread http://apache-ignite-users.70518.x6.nabble.com/How-to-use-rest-api-to-put-an-object-into-cache-td5897.html already discussed possible ways to solve this problem. On Wed, Jul 13, 2016 at 5:48 PM, Jason wrote: > seems that REST API only support simple key/value, like stri

Re: How does AffinityKey mapped?

Thanks for reply. Just confirm I understand right. IgniteCache orgCache IgniteCache comCache IgniteCache empCache In these three caches. The partition function will use EmployeeKey.organizationId to decide which node to store Employee. If orgCache, comCache's key have the same value with Employ

Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0

logs.rar Please refer to the attachment. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/ArrayIndexOutOfBoundsException-happened-when-doing-qurey-in-version-1-6-0-tp6245p6287.html Sent from t

Re: Does Apache Ignite support streaming serialization while returning results from the compute grid?

Hi Mohamed, There is no such support, but sending 4GB as a job result doesn't sound like a good idea to me. This will take a lot of time and also you will lose the result if the sender node fails in the middle of the process. Are you sure this is required for your use case? Can you elaborate more

Re: How does AffinityKey mapped?

Hi, You actually collocate keys, not caches. Let me explain in more detail. The primary node for a key is defined in two steps: 1. Map cache key to affinity key. This is defined by @AffinityKeyMapped annotation of AffinityKeyMapper. If none of them is provided, the affinity key is the same as cac

Re: Update performance

Hi Ionut, Yes, this is correct. Transaction is thread-local and I don't think what you propose is feasible, because someone has to coordinate the transaction. Map-reduce approach works for data processing, but not for transactional updates. I understand that you don't know the keys in advance, bu

Re: Cache EntryProcessor.process is getting invoked twice when cache.invoke() method is called within Transaction, in atomic mode its invoked once.

Hi, There is a ticket for this issue: https://issues.apache.org/jira/browse/IGNITE-3471. It should be fixed soon. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Cache-EntryProcessor-process-is-getting-invoked-twice-when-cache-invoke-method-is-called-withi

Re: Understanding data store and partitioning

Hi, You don't need to load all person IDs when loading the data. The loadCache() implementation can use Affinity API to get the array of local partition IDs and query the DB based on this IDs. With this approach each node will load only those rows that has to be stored locally. See the second code

Re: Iterating through a BinaryObject cache fails

This exception means that there was an attempt of BinaryObject deserialization, but there is no class definition for this BinaryObject. More specifically, it tries to find the mapping between type ID and class name, which is stored in the system cache and on the local FS when the object is serializ

Re: Failed to wait for initial partition map exchange

Hi Jason, There are a lot of possible reasons for that. Most likely something bad is happening (assertion, out of memory error, etc.) which freezes the cluster for some reason. I would recommend to collect full log files and thread dumps from all the nodes (servers and clients) and investigate the

Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0

I need the trace of ArrayIndexOutOfBoundsException which is the root cause. The log you provided shows 'caused by' line, but then it should also have the actual trace for it. It's fine if you simply attach the full log file. -Val -- View this message in context: http://apache-ignite-users.7051

Re: How does AffinityKey mapped?

hi I still have some confuse. If there are three cache like following. Which cache will empCache collocate with? orgCache or comCache? Since I don't see any other configuration in java example org.apache.ignite.examples.binary.datagrid.CacheClientBinaryQueryExample except @AffinityKeyMapped. Why E

Re: How does AffinityKey mapped?

Hello, You can to use both key with affinity part. For example: public class cache1Key { private int uniqueId1; @AffinityKeyMapped private int DEV_LV; } public class cache2Key { private int uniqueId2; @AffinityKeyMapped private int DEV_LV; } On Wed, Jul 13, 2016 at 4:

Re: Iterating through a BinaryObject cache fails

Just bringing up this discussion back up again since I have been experiencing the exception mentioned on this thread again. Caused by: class org.apache.ignite.IgniteCheckedException: Class definition was not found at marshaller cache and local file. [id=-1556878003, file=E:\ApacheIgnite\apache-ign

Re: How does AffinityKey mapped?

Thanks for reply I have another question. There are two tables. CREATE TABLE T1 ( LAT_IDSMALLINT, DEV_IDVARCHAR(20), DEV_LVSMALLINT ); CREATE TABLE T2 ( THE_DATE DATE, DEV_ID VARCHAR(20), DEV_LV SMALLINT ); SQL query: SELECT t

Re: Understanding data store and partitioning

Thanks for the follow up > Cache is a key-value store at first. Yes you can to use ID of record (from > database), as key of cache. Let's take an example of a Person table in an RDBMS database Person Id | Name | Address | Age | Date Of Birth Assuming "Person Id" is our cache key, to implement

Re: Understanding data store and partitioning

Hi, On Wed, Jul 13, 2016 at 3:48 PM, pragmaticbigdata wrote: > > You can find partition number using: affinity.partition(key) > > My question was - to get the partition id we need the cache key. When doing > the initial load into ignite we don't have the cache key. Does that mean we > cannot hav

Re: Understanding data store and partitioning

> You can find partition number using: affinity.partition(key) My question was - to get the partition id we need the cache key. When doing the initial load into ignite we don't have the cache key. Does that mean we cannot have an optimized data loading (i.e. partition aware data loading)? > Yes

Re: Cache EntryProcessor.process is getting invoked twice when cache.invoke() method is called within Transaction, in atomic mode its invoked once.

I am trying to understand this behavior of the entry processor. I could see that the entry processor is called on all server nodes including the backup nodes and the client node. The entry processor is executed on the client node when it is invoked from a transaction irrespective of the isolation l

Re: Understanding data store and partitioning

Hello, On Wed, Jul 13, 2016 at 2:26 PM, pragmaticbigdata wrote: > Following the documentation on data loading > I have some questions > with regards to ignite version 1.6 > > 1. How does ignite derive the partition id from the cache key? Wh

Re: How to troubleshoot a slow client node get()

Tracel, can you please also take threaddumps of both client and server JVMs when you see that get() takes too long? --Yakov 2016-07-13 12:33 GMT+03:00 tracel : > Thanks yakov, > I am using a Linux box. > The delay can also be observed after leaving client idle for some 3 to 10 > minutes. > > I w

Re: Issue with concurrent users on Ignite 1.6.0 ODBC

Agneeswaran, What I can see is that SEGFAULT caused by the misuse of the SQLGetDiagRec - our current implementation of the function assumes that TextLengthPtr could not be NULL. Fix that and you are going to get some meaningful error message that we could analyze further. From the core dump it loo

Re: How to troubleshoot a slow client node get()

On Wed, Jul 13, 2016 at 11:23 AM, tracel wrote: > thanks dsetrakyan, > > Why use System.out.println()? > I want to make sure that there is no overhead associated with log.info(). Can you check? > I have added the System.out.println(), and keep the log.info() just for > comparison: > >

Re: How does AffinityKey mapped?

Hi, In addition to answering Andrew On Wed, Jul 13, 2016 at 3:08 PM, Andrey Gura wrote: > Hi, > > It is responsibility of AffinityKeyMapper interface implementation. By > default Ignite uses GridCacheDefaultAffinityKeyMapper class for this > purposes. But you can provide own mapper implementat

Re: How does AffinityKey mapped?

Hi, It is responsibility of AffinityKeyMapper interface implementation. By default Ignite uses GridCacheDefaultAffinityKeyMapper class for this purposes. But you can provide own mapper implementation using CacheConfiguration.setAffinityMapper() method. On Wed, Jul 13, 2016 at 11:58 AM, November

Understanding data store and partitioning

Following the documentation on data loading I have some questions with regards to ignite version 1.6 1. How does ignite derive the partition id from the cache key? What is the relation between the partition id and the affinity key? 2. Partitio

Does the HTTP REST API support the binary value?

seems that REST API only support simple key/value, like string. If the value is a class with multiple columns, and some of them are used to do SQL query. how to send this kind of data by using REST API? Thanks, -Jason -- View this message in context: http://apache-ignite-users.70518.x6.nabble

Re: How to troubleshoot a slow client node get()

Thanks yakov, I am using a Linux box. The delay can also be observed after leaving client idle for some 3 to 10 minutes. I will try disabling the shared memory communication, Thanks! -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-troubleshoot-a-slow-clie

Re: How to troubleshoot a slow client node get()

What OS do do you use? Is this delay observed only initially or can also be observed after leaving client idle after doing some work? Can you please disable shared memory communication in case you use Linux or Mac - set org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi#setSharedMemoryPor

How does AffinityKey mapped?

Hi I have read http://apacheignite.gridgain.org/docs/affinity-collocation and java example. I don't know how AffinityKey map? Like following code in java example. How does ignite know EmployeeKey's organizationId should collocate with Organization's id (not other fields). I can't find anything m

Re: How to troubleshoot a slow client node get()

After done some more testings, I have these findings so far: - The slow get() symptom only found when the client node and the server node are started on the SAME machine, I cannot reproduce the symptom when client node and server node were started on their own machines. - The symptom seems to occ

Re: How to troubleshoot a slow client node get()

thanks dsetrakyan, Why use System.out.println()? I have added the System.out.println(), and keep the log.info() just for comparison: log.info("### Before get()"); System.out.println("##~ Before get()"); Vendor vendor = cache.get(vendorCode);

Re: Issue with concurrent users on Ignite 1.6.0 ODBC

Hi Igor, Please find the attached lib files, Thanks, Agneeswaran lib1.zip lib2.zip lib3.zip

回复: Ignite used memory 7 times greater than imported data

Now， because I created my cache in java code, so changing the config.xml doesn't work. I already change the java code of creating my cache to change the backups to zero from one: CacheConfiguration cfg = CacheConfig.cache(cacheName, pojoStoreFactory); //cfg.setBackups(1)

Re: Update performance

Hi Val, Lately I was using optimistic serializable which has better performance in my test cases. I see two problems with EntryProcessor: 1. When the "where" condition is generic I don't have the keys therefore an initial step is required to move all the keys in the client. 2. In my tests putAl

Re: How to troubleshoot a slow client node get()

Can you replace log.info() with System.out.println() in your test? On Wed, Jul 13, 2016 at 10:21 AM, tracel wrote: > I have an Ignite (1.5.0.final) cache client node started in a Tomcat > 8.0.32, > the client node connects to a server node started on the same machine. > > Sometimes the get() nee

Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0

What trace do you need? The log files in work/log? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/ArrayIndexOutOfBoundsException-happened-when-doing-qurey-in-version-1-6-0-tp6245p6253.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Does Apache Ignite support streaming serialization while returning results from the compute grid?

Hi, Currently we are using Apache Ignite for caching huge amount of data (> 500 GB) and performing some computations over it. One of our computations required returning big chunk of data (> 4GB after serialization) from the compute node to the requester, which caused integer overflow while seri

Failed to wait for initial partition map exchange

hi Ignite Devs, I'm running an Ignite cluster with ~20 server nodes (.net version) and running ~120 client nodes that are started from a .net program from different machines with those of server nodes. All the machines are in Windows server 2012. In the config of server node: 1. use the static h

How to troubleshoot a slow client node get()

I have an Ignite (1.5.0.final) cache client node started in a Tomcat 8.0.32, the client node connects to a server node started on the same machine. Sometimes the get() need some 5 seconds, while most of the other get() need almost no time. I wonder how the 5 seconds was spent, how I can troublesho