Re: Issue with Java 8 datatype LocalDate while using IgniteRDD

2016-04-29 Thread vkulichenko
Hi Vij, Looks like the issue was reproduced, but it's a bit more complicated and the reason is not clear yet. Here is the ticket for this: https://issues.apache.org/jira/browse/IGNITE-3077 -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Issue-with-Java-8-d

Re: Ignite support for .net and c++

2016-04-29 Thread vkulichenko
Hi Murthy, >From what I grasped from your previous posts, Ignite definitely fits your use case, and I'm not aware of any other products that provide Java, .NET and C++ support at the same time. .NET and especially C++ support are comparatively new features of Ignite, so bugs and usability issues

Re: HDFS Caching

2016-04-29 Thread vkulichenko
Hi, As far as I know, IGFS can redirect to secondary file system not only to read from it, but also for integrity purposes (e.g., to check if the file in secondary FS was updated directly, without updating IGFS). In any case, the data itself will be read from memory if it is there. I would try to

Ignite support for .net and c++

2016-04-29 Thread Murthy Kakarlamudi
Ignite gurus, My application consists of .net and c++ components. Currently we use shared database pattern to exchange data across the components. As our volume is pretty huge, we were looking for an in-memory solution and Ignite Data Grid with write behind support seemed like a perfect match

Re: Running gridgain yardstick

2016-04-29 Thread vkulichenko
Hi, How do you build the project? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Running-gridgain-yardstick-tp4559p4710.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Client fails to connect - joinTimeout vs networkTimeout

2016-04-29 Thread vkulichenko
Caches always use affinity, it defines how the data is distributed across nodes. If you don't explicitly provide it in the configuration, RendezvousAffinityFunction will be used with excludeNeighbors=false. So if you want to enable this feature, you have to specify this in the configuration. -Val

Re: Number of partitions of IgniteRDD

2016-04-29 Thread vkulichenko
Vij, This is because you're checking partitions for result DataFrame. IgniteRDD queries Ignite directly, gets the result back and wraps it into the DataFrame. If you do any transformations with this DataFrame, they are not parallelized and are done on the driver. Thus only one partition returned.

Re: Client fails to connect - joinTimeout vs networkTimeout

2016-04-29 Thread bintisepaha
Val, thanks a lot. Will this also work if the caches do not use affinity? We are trying not to use affinity because our data is very skewed. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Client-fails-to-connect-joinTimeout-vs-networkTimeout-tp4419p4706.html Sen

Re: Affinitykey is not working

2016-04-29 Thread Alexey Goncharuk
Hi, Scala does not automatically place annotations to generated fields, you need to use the annotation as follows: @(AffinityKeyMapped @field) val marketSectorId:Int = 0

Re: Error starting c++ client node using 1.6

2016-04-29 Thread Igor Sapego
It seems like you are trying to use client node binary from Ignite 1.5 for Ignite 1.6. You should build client node binary for the 1.6 instead to use with Ignite 1.6 as they are not cross-compatible. Best Regards, Igor On Fri, Apr 29, 2016 at 8:39 PM, Igor Sapego wrote: > Hi, > > Where have you

Re: Error starting c++ client node using 1.6

2016-04-29 Thread Igor Sapego
Hi, Where have you got your C++ client node binary from? Best Regards, Igor On Fri, Apr 29, 2016 at 8:32 PM, Murthy Kakarlamudi wrote: > Hi All, > I downloaded the latest 1.6 binary from latest builds. I am trying to > start a node from c++ and getting the below error. > > An error occurre

Error starting c++ client node using 1.6

2016-04-29 Thread Murthy Kakarlamudi
Hi All, I downloaded the latest 1.6 binary from latest builds. I am trying to start a node from c++ and getting the below error. An error occurred: Failed to initialize JVM [errCls=java.lang.NoSuchMethodError, errMsg=executeNative] The same c++ node starts fine if I point my IGNITE_HOME to 1

Re: Number of partitions of IgniteRDD

2016-04-29 Thread vijayendra bhati
My bad !!! Yea you are right.But now the problem is when I get DataFrame by using below code and get JavaRDD from it , its number of partitions is 1 and its of type  MapPartitionsRDD. String sql = "select simulationUUID,stockReturn from STOCKSIMULATIONRETURNSVAL where businessDate = ? and symbol

HDFS Caching

2016-04-29 Thread barham
I'm running Ignite 1.5.0 Hadoop Accelerator version on top of CDH 5. I'm trying to write my own SecondaryFileSystem, but as a first step, I created one that just funnels all of the calls down to the IgniteHadoopIgfsSecondaryFileSystem and I just log out every time one of my methods is called. I'm

Re: Issue with Java 8 datatype LocalDate while using IgniteRDD

2016-04-29 Thread vijayendra bhati
IgniteRDD makes the value of type struct instead of LocalDate and hence below exception comes - Caused by: scala.MatchError: 2016-03-17 (of class java.time.LocalDate) at org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:255) at org.

Re: Running gridgain yardstick

2016-04-29 Thread akritibahal91
Hi, When I do bin/benchmark-run-all.sh config/benchmark.properties, I get the following but I'm still unable to run the benchmarks. My ssh is open. <10:33:37> Starting server config '...2 -sn EchoServer -dn EchoBenchmark -ds EchoServer-2-threads' on localhost with id=0 Password:debug3: Received S

Re: SQL Aliases are not interpreted correctly

2016-04-29 Thread jan.swaelens
Hello, Thanks works like a charm! Up to the next level in my experiment. br jan -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/SQL-Aliases-are-not-interpreted-correctly-tp4281p4692.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Number of partitions of IgniteRDD

2016-04-29 Thread Vladimir Ozerov
Hi Vij, I see method "getPartitions" in IgniteRDD, not "getNumPartitions". Please confirm that we are talking about the same thing. Anyway, logic of this method is extremely straightforward - it simply call Ignite.affinity("name_of_your_cache").partitions() method, so it should return actual numb

Re: Number of partitions of IgniteRDD

2016-04-29 Thread vijayendra bhati
Yes its Spark RDD's standard method, but it has been overridden in IgniteRDD. Regards,Vij On Friday, April 29, 2016 5:25 PM, Vladimir Ozerov wrote: Hi Vij, I am not quite uderstand where does method "getNumPartitions" came from. Is it on standard Spark API? I do not see it on org.apach

Re: Affinitykey is not working

2016-04-29 Thread Vladimir Ozerov
Hi, Could you please explain how do you detect a node to which key is mapped? Do you use Affinity API? Vladimir. On Fri, Apr 29, 2016 at 11:48 AM, nikhilknk wrote: > I used the below ToleranceCacheKey as the key . I want to keep all the > keys > whose marketSectorId is same in the same node

Re: Issue with Java 8 datatype LocalDate while using IgniteRDD

2016-04-29 Thread Vladimir Ozerov
Hi Vij, Do you see any exception or some other kind of error? Please provide more error description. Vladimir. On Fri, Apr 29, 2016 at 2:47 PM, vijayendra bhati wrote: > Hi Guys, > > I am trying to store a object which contains object of type LocalDate > datatype of Java8 time's API > I am fac

Re: Number of partitions of IgniteRDD

2016-04-29 Thread Vladimir Ozerov
Hi Vij, I am not quite uderstand where does method "getNumPartitions" came from. Is it on standard Spark API? I do not see it on* org.apache.spark.api.java.JavaRDD* class. Vladimir. On Fri, Apr 29, 2016 at 7:50 AM, vijayendra bhati wrote: > Hi Val, > > I am creating DataFrame using below code

Issue with Java 8 datatype LocalDate while using IgniteRDD

2016-04-29 Thread vijayendra bhati
Hi Guys,  I am trying to store a object which contains object of type LocalDate datatype of Java8 time's APII am facing issues over it while working with IgniteRDDLooks like LocalDate is not handle in IgniteRDD and may be in Spark as well Anybody can help here ? Regards,Vij

Affinitykey is not working

2016-04-29 Thread nikhilknk
I used the below ToleranceCacheKey as the key . I want to keep all the keys whose marketSectorId is same in the same node . So I kept annotation "@AffinityKeyMapped" for marketSectorId . I started 3 nodes of ignite cluster but the instruments of same marketSectorId are sahred among three nodes

Re: Ignite Installation with Spark under CDH

2016-04-29 Thread Vladimir Ozerov
Hi Michael, Ok, so it looks like the process didn't have enough heap. Thank you for your inputs about CDH configuration. We will improve our documentation based on this. Vladimir On Thu, Apr 28, 2016 at 5:15 PM, mdolgonos wrote: > Vladimir, > > I fixed this by changing the way I start Ignite b