Gianluca, have you had any luck with S3 storage of logs for history server?
Best, Vladimir On Tue, Jun 16, 2015 at 6:53 PM, Gianluca Privitera < gianluca.privite...@studio.unibo.it> wrote: > It gives me an exception with > org.apache.spark.deploy.history.FsHistoryProvider , a problem with the > file system. I can reproduce the exception if you want. > It perfectly works if I give a local path, I tested it in 1.3.0 version. > > Gianluca > > On 16 Jun 2015, at 15:08, Akhil Das <ak...@sigmoidanalytics.com> wrote: > > Not quiet sure, but try pointing the spark.history.fs.logDirectory to your > s3 > > Thanks > Best Regards > > On Tue, Jun 16, 2015 at 6:26 PM, Gianluca Privitera < > gianluca.privite...@studio.unibo.it> wrote: > >> In Spark website it’s stated in the View After the Fact section ( >> https://spark.apache.org/docs/latest/monitoring.html) that you can point >> the start-history-server.sh script to a directory in order do view the Web >> UI using the logs as data source. >> >> Is it possible to point that script to S3? Maybe from a EC2 instance? >> >> Thanks, >> >> Gianluca >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> > >
