Following Hadoop conventions, Spark won't overwrite an existing directory. You need to provide a unique output path every time you run the program, or delete or rename the target directory before you run the job.
dean Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Mon, Aug 10, 2015 at 7:08 AM, Yasemin Kaya <godo...@gmail.com> wrote: > Hi, > > I have EC2 cluster, and am using spark 1.3, yarn and HDFS . When i submit > at local there is no problem , but i run at cluster, saveAsTextFile doesn't > work."*It says me User class threw exception: Output directory > hdfs://172.31.42.10:54310/...../weblogReadResult > <http://172.31.42.10:54310/...../weblogReadResult> already exists*" > > Is there anyone can help me about this issue ? > > Best, > yasemin > > > > -- > hiç ender hiç >
