On 13 January 2017 at 13:55, Anahita Talebi <anahita.t.am...@gmail.com> wrote: > Hi, > > Thanks for your answer. > > I have chose "Spark" in the "job type". There is not any option where we can > choose the version. How I can choose different version?
There's "Preemptible workers, bucket, network, version, initialization, & access options" link just above the "Create" and "Cancel" buttons on the "Create a cluster" page. When you click it, you'll find "Image version" field where you can enter the image version. Dataproc versions: * 1.1 would be Spark 2.0.2, * 1.0 includes Spark 1.6.2 More about versions can be found here: https://cloud.google.com/dataproc/docs/concepts/dataproc-versions Cheers, Dinko > > Thanks, > Anahita > > > On Thu, Jan 12, 2017 at 6:39 PM, A Shaikh <shaikh.af...@gmail.com> wrote: >> >> You may have tested this code on Spark version on your local machine >> version of which may be different to whats in Google Cloud Storage. >> You need to select appropraite Spark version when you submit your job. >> >> On 12 January 2017 at 15:51, Anahita Talebi <anahita.t.am...@gmail.com> >> wrote: >>> >>> Dear all, >>> >>> I am trying to run a .jar file as a job using submit job in google cloud >>> console. >>> https://cloud.google.com/dataproc/docs/guides/submit-job >>> >>> I actually ran the spark code on my local computer to generate a .jar >>> file. Then in the Argument folder, I give the value of the arguments that I >>> used in the spark code. One of the argument is training data set that I put >>> in the same bucket that I save my .jar file. In the bucket, I put only the >>> .jar file, training dataset and testing dataset. >>> >>> Main class or jar >>> gs://Anahita/test.jar >>> >>> Arguments >>> >>> --lambda=.001 >>> --eta=1.0 >>> --trainFile=gs://Anahita/small_train.dat >>> --testFile=gs://Anahita/small_test.dat >>> >>> The problem is that when I run the job I get the following error and >>> actually it cannot read my training and testing data sets. >>> >>> Exception in thread "main" java.lang.NoSuchMethodError: >>> org.apache.spark.rdd.RDD.coalesce(IZLscala/math/Ordering;)Lorg/apache/spark/rdd/RDD; >>> >>> Can anyone help me how I can solve this problem? >>> >>> Thanks, >>> >>> Anahita >>> >>> >> > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
