Here is one blog illustrating how to use Spark on Alluxio for this purpose.
Hope it will help:
http://www.alluxio.com/2016/04/getting-started-with-alluxio-and-spark/
On Mon, Jul 18, 2016 at 6:36 AM, Gene Pang wrote:
> Hi,
>
> If you want to use Alluxio with Spark 2.x, it is recommended to write
Hi,
If you want to use Alluxio with Spark 2.x, it is recommended to write to
and read from Alluxio with files. You can save an RDD with saveAsObjectFile
with an Alluxio path (alluxio://host:port/path/to/file), and you can read
that file from any other Spark job. Here is additional information on h
Hi All,
I have some questions about OFF_HEAP Caching. In Spark 1.X when we use
rdd.persist(StorageLevel.OFF_HEAP),that means rdd caching in Tachyon(Alluxio).
However,in Spark 2.X,we can directly use OFF_HEAP For Caching
(https://issues.apache.org/jira/browse/SPARK-13992?jql=project%20%3D%20SPA
