Gianluca,
have you had any luck with S3 storage of logs for history server?
Best,
Vladimir
On Tue, Jun 16, 2015 at 6:53 PM, Gianluca Privitera <
gianluca.privite...@studio.unibo.it> wrote:
> It gives me an exception with
> org.apache.spark.deploy.history.FsHistoryProvider , a problem with the
It gives me an exception with org.apache.spark.deploy.history.FsHistoryProvider
, a problem with the file system. I can reproduce the exception if you want.
It perfectly works if I give a local path, I tested it in 1.3.0 version.
Gianluca
On 16 Jun 2015, at 15:08, Akhil Das
mailto:ak...@sigmoid
Not quiet sure, but try pointing the spark.history.fs.logDirectory to your
s3
Thanks
Best Regards
On Tue, Jun 16, 2015 at 6:26 PM, Gianluca Privitera <
gianluca.privite...@studio.unibo.it> wrote:
> In Spark website it’s stated in the View After the Fact section (
> https://spark.apache.org/docs/
In Spark website it’s stated in the View After the Fact section
(https://spark.apache.org/docs/latest/monitoring.html) that you can point the
start-history-server.sh script to a directory in order do view the Web UI using
the logs as data source.
Is it possible to point that script to S3? Maybe
