Valentin Kulichenko created IGNITE-13209:
--------------------------------------------
Summary: JavaIgniteCatalogExample doesn't work with a standalone
Spark cluster
Key: IGNITE-13209
URL: https://issues.apache.org/jira/browse/IGNITE-13209
Project: Ignite
Issue Type: Bug
Components: spark
Affects Versions: 2.8.1
Reporter: Valentin Kulichenko
To reproduce the issue:
# Start Spark master and slave as described here:
[http://spark.apache.org/docs/latest/spark-standalone.html]
# Change the master URL in the {{JavaIgniteCatalogExample}} from "local" to
the one just started.
# Run the example.
Updated code that creates the {{IgniteSparkSession}}:
{code:java}
String libs =
"/Users/vkulichenko/GridGain/releases/apache-ignite-2.8.1-bin/libs";
IgniteSparkSession igniteSession = IgniteSparkSession.builder()
.appName("Spark Ignite catalog example")
.master("spark://Valentin-Kulichenko-MacBook-Pro-1772.local:7077")
.config("spark.executor.instances", "2")
.config("spark.executor.extraClassPath", libs + "/*" + ":" + libs +
"/ignite-spark/*:" + libs + "/ignite-spring/*")
.igniteConfig(CONFIG)
.getOrCreate();
{code}
Execution fails with this exception:
{noformat}
[2020-07-02 15:50:27,627][ERROR][task-result-getter-3][TaskSetManager] Task 0
in stage 0.0 failed 4 times; aborting job
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to
stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task 0.3 in stage 0.0 (TID 3, 10.0.0.11, executor 0): class
org.apache.ignite.IgniteIllegalStateException: Ignite instance with provided
name doesn't exist. Did you call Ignition.start(..) to start an Ignite
instance? [name=testing]
at org.apache.ignite.internal.IgnitionEx.grid(IgnitionEx.java:1351)
at org.apache.ignite.Ignition.ignite(Ignition.java:528)
at org.apache.ignite.spark.impl.package$.ignite(package.scala:65)
at
org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:238)
at
org.apache.ignite.spark.impl.IgniteRelationProvider$$anonfun$configProvider$1$2.apply(IgniteRelationProvider.scala:235)
at org.apache.ignite.spark.Once.apply(IgniteContext.scala:222)
at org.apache.ignite.spark.IgniteContext.ignite(IgniteContext.scala:144)
at
org.apache.ignite.spark.impl.IgniteSQLDataFrameRDD.compute(IgniteSQLDataFrameRDD.scala:65)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:109)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
