Re: How to encrypt AK and SK using Livy restAPI to submit sparkJob

> > Hi, > > I am using Livy RestApI to submit spark job. I used s3a to replace HDFS. > I have to write fs.s3a.access.key and fs.s3a.sercet.key directly in > core-site.xml, as there is no these conf param in Livy API. > How could I encrypt my AK and SK? > > Yours. > Jane >

How to encrypt AK and SK using Livy restAPI to submit sparkJob

Hi, I am using Livy RestApI to submit spark job. I used s3a to replace HDFS. I have to write fs.s3a.access.key and fs.s3a.sercet.key directly in core-site.xml, as there is no these conf param in Livy API. How could I encrypt my AK and SK? Yours. Jane

Re: [spark on yarn] spark on yarn without DFS

issue could be something else too - so a stacktrace or error message >> could help in understanding the problem. >> >> >> >> On Mon, May 20, 2019, 07:20 Huizhe Wang wrote: >> >>> Hi, >>> >>> I wanna to use Spark on Yarn without HDFS

[spark on yarn] spark on yarn without DFS

Hi, I wanna to use Spark on Yarn without HDFS.I store my resource in AWS and using s3a to get them. However, when I use stop-dfs.sh stoped Namenode and DataNode. I got an error when using yarn cluster mode. Could I using yarn without start DFS, how could I use this mode? Yours, Jane