Hi,
If you are looking for how to run Spark on Alluxio (formerly Tachyon),
here is the documentation from Alluxio doc site:
http://www.alluxio.org/docs/master/en/Running-Spark-on-Alluxio.html
It still works for Spark 2.x.
Alluxio team also published articles on when and why running Spark (2.x)
wi
It backed the "OFF_HEAP" storage level for RDDs. That's not quite the
same thing that off-heap Tungsten allocation refers to.
It's also worth pointing out that things like HDFS also can put data
into memory already.
On Mon, Sep 19, 2016 at 7:48 PM, Richard Catlin
wrote:
> Here is my understandin
Here is my understanding.
Spark used Tachyon as an off-heap solution for RDDs. In certain situations, it
would alleviate Garbage Collection or the RDDs.
Tungsten, Spark 2’s off-heap (columnar format) is much more efficient and used
as the default. Alluvio no longer makes sense for this use.
Hi folks,
What has happened with Tachyon / Alluxio in Spark 2? Doc doesn't mention it
no longer.
--
Oleksiy Dyagilev
