Why not setup HDFS? Thanks Best Regards
On Thu, Mar 5, 2015 at 4:03 PM, didmar <marin.did...@gmail.com> wrote: > Hi, > > I'm having a problem involving file permissions on the local filesystem. > > On a first machine, I have two different users : > - launcher, which launches my job from an uber jar file > - spark, which runs the master > On a second machine, I have a user spark (same uid/gid as the other) which > runs the worker > > Results are written in a shared folder /home/spark/output/ (same path for > both machines) which is owned by spark and has 777 permissions > > When I run a job that saves a text file to /home/spark/output/result/, > launcher creates the result folder and subfolders with 775 permissions. > The problem is that the worker, which runs has user spark, cannot write the > results : > > > > I tried to add the sticky bit to /home/spark/output/, but this did not > suffice : it creates a part-00000, starts filling it but then the following > error occurs > > > > Is there a way to force Spark (the driver part I suppose ?) to create all > these folders with 777 permissions instead of 775 ? Or maybe there is > another way ? > > Thanks, > Didier > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Managing-permissions-when-saving-as-text-file-tp21928.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
