Noritaka Sekiyama created SPARK-18432:
-----------------------------------------
Summary: Fix HDFS block size in programming guide
Key: SPARK-18432
URL: https://issues.apache.org/jira/browse/SPARK-18432
Project: Spark
Issue Type: Documentation
Components: Documentation
Affects Versions: 2.0.1
Reporter: Noritaka Sekiyama
Priority: Minor
http://spark.apache.org/docs/latest/programming-guide.html
"By default, Spark creates one partition for each block of the file (blocks
being 64MB by default in HDFS)"
Currently default block size in HDFS is 128MB.
The default value has been already increased in Hadoop 2.2.0 (the oldest
supported version of Spark). https://issues.apache.org/jira/browse/HDFS-4053
Since it looks confusing explanation, I'd like to fix the value from 64MB to
128MB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
