hi Andy Assuming you are running Spark with YARN, then I would recommend deploying Alluxio in the same YARN cluster if you are looking for best performance. Alluxio can also be deployed separated as a standalone service, but in that case, you may need to transfer data from Alluxio cluster to your Spark/YARN cluster.
Here is the documentation <https://docs.alluxio.io/os/user/1.8/en/deploy/Running-Alluxio-On-Yarn.html?utm_source=spark> about deploying Alluxio with YARN. - Bin On Thu, May 9, 2019 at 4:19 AM u9g <lwx371...@163.com> wrote: > Hey, > > I want to speed up the Spark task running in the Yarn cluster through > Alluxio. Is Alluxio recommended to run in the same yarn cluster on the yarn > mode? Should I deploy Alluxio independently on the nodes of the yarn > cluster? Or deploy a cluster separately? > Best, > Andy Li > > > >
