Hi, I'm having a problem involving file permissions on the local filesystem.
On a first machine, I have two different users : - launcher, which launches my job from an uber jar file - spark, which runs the master On a second machine, I have a user spark (same uid/gid as the other) which runs the worker Results are written in a shared folder /home/spark/output/ (same path for both machines) which is owned by spark and has 777 permissions When I run a job that saves a text file to /home/spark/output/result/, launcher creates the result folder and subfolders with 775 permissions. The problem is that the worker, which runs has user spark, cannot write the results : I tried to add the sticky bit to /home/spark/output/, but this did not suffice : it creates a part-00000, starts filling it but then the following error occurs Is there a way to force Spark (the driver part I suppose ?) to create all these folders with 777 permissions instead of 775 ? Or maybe there is another way ? Thanks, Didier -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Managing-permissions-when-saving-as-text-file-tp21928.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
