Anyone has idea on this ?
Thx
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Understanding-spark-operation-pipeline-and-block-storage-tp18201p19263.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
Hi Cheng,
You are right. =)
I checked your article a few days ago. I have some further questions:
According to the article, the following code takes the spatial complexity
o(1).
val lines = spark.textFile("hdfs://")
val errors = lines.filter(_.startsWith("ERROR"))
val messages = errors.map(_.sp
On 11/6/14 1:39 AM, Hao Ren wrote:
Hi,
I would like to understand the pipeline of spark's operation(transformation
and action) and some details on block storage.
Let's consider the following code:
val rdd1 = SparkContext.textFile("hdfs://...")
rdd1.map(func1).map(func2).count
For example, we
Hey, guys
Feel free to ask for more details if my questions are not clear.
Any insight here ?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Understanding-spark-operation-pipeline-and-block-storage-tp18201p18496.html
Sent from the Apache Spark User Lis
