Cos,
Ignite provides Java friendly RDD API. See JavaIgniteContext and
JavaIgniteRDD classes.
On Fri, Oct 9, 2015 at 12:26 AM, Konstantin Boudnik <c...@apache.org> wrote:
> Guys,
>
> I've tried to play a little bit with IgniteContext and the whole IgniteRDD
> stuff from the perspective of not touching Scala ever again. And here's
> what I
> have found: IgniteContext isn't usable from Java (or Groovy for that
> matter).
> And it isn't an attempt to critique Ignite's RDD implementation, because we
> have to follow the design patterns setup by the 3rd party platform, Spark
> in
> this case.
>
> If I want to submit a Java-based job into spark cluster I need to do
> something
> like this:
>
> import org.apache.ignite.spark.*
> import org.apache.spark.SparkConf
> import org.apache.spark.api.java.JavaSparkContext
>
> SparkConf sparkConf = new SparkConf().setAppName('SharedCache')
> sparkConf.setMaster('spark://master.spark.mydomain.com:7077')
> JavaSparkContext jsc = new JavaSparkContext(sparkConf)
>
> so far so good. Now, if I want to use Ignite in it, I should be doing
>
> def ic = new IgniteContext<T1, T2>(jsc, "spark-ignite-config.xml")
>
> and that's where I hit the wall, because JavaSparkContext isn't a subtype
> of
> SparkContext in Spark world, and I can not cast one to another.
>
> Well then, perhaps I can use SparkContext and hope for the best? Turns out
> I
> can not, because the use of the SparkContext gets me into the swamp of
> Scala
> type-system, with long-mnemonic names like Function1 and Tuple2 (I reckon
> the
> day will come soon, when they will have Function27 and so on). This leads
> me
> to believe the current implementation of IgniteRDD is only good to be used
> from Scala, unless I am completely wrong and don't know what I am talking
> about. Which might be quite possible, of course.
>
> My question is very simple: is there a way to use IgniteRDD from
> Java-language
> family, e.g. Java and Groovy, or there has to be JavaIgniteRDD
> implementation
> of it much like the state of things is in the Spark itself?
>
> Thanks for any feedback
> --
> Cos
>
>
--
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com
