Ah, missed that. Thanks!
It is pity to see how Scala based platform forces every downstream to treat
Java code like a second-class citizen. How nice ;(
Cos
On Fri, Oct 09, 2015 at 01:11PM, Andrey Gura wrote:
> Cos,
>
> Ignite provides Java friendly RDD API. See JavaIgniteContext and
> JavaIgniteRDD classes.
>
> On Fri, Oct 9, 2015 at 12:26 AM, Konstantin Boudnik <c...@apache.org> wrote:
>
> > Guys,
> >
> > I've tried to play a little bit with IgniteContext and the whole IgniteRDD
> > stuff from the perspective of not touching Scala ever again. And here's
> > what I
> > have found: IgniteContext isn't usable from Java (or Groovy for that
> > matter).
> > And it isn't an attempt to critique Ignite's RDD implementation, because we
> > have to follow the design patterns setup by the 3rd party platform, Spark
> > in
> > this case.
> >
> > If I want to submit a Java-based job into spark cluster I need to do
> > something
> > like this:
> >
> > import org.apache.ignite.spark.*
> > import org.apache.spark.SparkConf
> > import org.apache.spark.api.java.JavaSparkContext
> >
> > SparkConf sparkConf = new SparkConf().setAppName('SharedCache')
> > sparkConf.setMaster('spark://master.spark.mydomain.com:7077')
> > JavaSparkContext jsc = new JavaSparkContext(sparkConf)
> >
> > so far so good. Now, if I want to use Ignite in it, I should be doing
> >
> > def ic = new IgniteContext<T1, T2>(jsc, "spark-ignite-config.xml")
> >
> > and that's where I hit the wall, because JavaSparkContext isn't a subtype
> > of
> > SparkContext in Spark world, and I can not cast one to another.
> >
> > Well then, perhaps I can use SparkContext and hope for the best? Turns out
> > I
> > can not, because the use of the SparkContext gets me into the swamp of
> > Scala
> > type-system, with long-mnemonic names like Function1 and Tuple2 (I reckon
> > the
> > day will come soon, when they will have Function27 and so on). This leads
> > me
> > to believe the current implementation of IgniteRDD is only good to be used
> > from Scala, unless I am completely wrong and don't know what I am talking
> > about. Which might be quite possible, of course.
> >
> > My question is very simple: is there a way to use IgniteRDD from
> > Java-language
> > family, e.g. Java and Groovy, or there has to be JavaIgniteRDD
> > implementation
> > of it much like the state of things is in the Spark itself?
> >
> > Thanks for any feedback
> > --
> > Cos
> >
> >
>
>
> --
> Andrey Gura
> GridGain Systems, Inc.
> www.gridgain.com
