I recently installed spark standalone through cloudera manager on my CDH
5.2 cluster. CDH 5.2 is runing on CentOS release 6.6. The version of
spark again through Cloudera is 1.1. It is standalone.
I have a file in hdfs in /tmp/testfile.txt.
So what I do is i run spark-shell:
scala> val source
output is coming on
> spark-shell. If yes, then most probably in your case, there might be some
> issue with your configuration files. It will be helpful if you can paste
> the contents of the config files you edited.
>
> On Thu, Nov 20, 2014 at 5:45 AM, Anson Abraham
> wrote:
>
Sorry meant cdh 5.2 w/ spark 1.1.
On Wed, Nov 19, 2014, 17:41 Anson Abraham wrote:
> yeah CDH distribution (1.1).
>
> On Wed Nov 19 2014 at 5:29:39 PM Marcelo Vanzin
> wrote:
>
>> On Wed, Nov 19, 2014 at 2:13 PM, Anson Abraham
>> wrote:
>> > yeah but in
yeah CDH distribution (1.1).
On Wed Nov 19 2014 at 5:29:39 PM Marcelo Vanzin wrote:
> On Wed, Nov 19, 2014 at 2:13 PM, Anson Abraham
> wrote:
> > yeah but in this case i'm not building any files. just deployed out
> config
> > files in CDH5.2 and initiated a sp
d SPARK-4048, but I digress.)
>
>
> On Tue, Nov 18, 2014 at 1:59 PM, Anson Abraham
> wrote:
> > I'm essentially loading a file and saving output to another location:
> >
> > val source = sc.textFile("/tmp/testfile.txt")
> > source.saveAsTextFile(&q
Question ... when you mean different versions, different versions of
dependency files? what are the dependency files for spark?
On Tue Nov 18 2014 at 5:27:18 PM Anson Abraham
wrote:
> when cdh cluster was running, i did not set up spark role. When I did for
> the first time, it was work
mail.com> wrote:
> It can be a serialization issue.
> Happens when there are different versions installed on the same system.
> What do you mean by the first time you installed and tested it out?
>
> On Wed, Nov 19, 2014 at 3:29 AM, Anson Abraham
> wrote:
>
>> I&#
I'm essentially loading a file and saving output to another location:
val source = sc.textFile("/tmp/testfile.txt")
source.saveAsTextFile("/tmp/testsparkoutput")
when i do so, i'm hitting this error:
14/11/18 21:15:08 INFO DAGScheduler: Failed to run saveAsTextFile at
:15
org.apache.spark.SparkEx
