Re: How to configure alluxio cluster with spark in yarn

2019-05-16 Thread Bin Fan
hi Andy Assuming you are running Spark with YARN, then I would recommend deploying Alluxio in the same YARN cluster if you are looking for best performance. Alluxio can also be deployed separated as a standalone service, but in that case, you may need to transfer data from Alluxio cluster to your

How to configure alluxio cluster with spark in yarn

2019-05-09 Thread u9g
Hey, I want to speed up the Spark task running in the Yarn cluster through Alluxio. Is Alluxio recommended to run in the same yarn cluster on the yarn mode? Should I deploy Alluxio independently on the nodes of the yarn cluster? Or deploy a cluster separately? Best, Andy Li