[
https://issues.apache.org/jira/browse/SPARK-18188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davies Liu updated SPARK-18188:
-------------------------------
Description:
There is an understanding issue for a long time:
https://issues.apache.org/jira/browse/SPARK-4105, without any checksum for the
blocks, it's very hard for us to identify where is the bug came from.
Shuffle blocks are compressed separate (have checksum in it), but broadcast
blocks are compressed together, we should add checksum for each of separately.
was:
There is an understanding issue for a long time:
https://issues.apache.org/jira/browse/SPARK-4105, without any checksum for the
blocks, it's very hard for us to identify where is the bug came from.
We should have a way the check a block from remote node or disk that is correct
or not.
> Add checksum for block of broadcast
> -----------------------------------
>
> Key: SPARK-18188
> URL: https://issues.apache.org/jira/browse/SPARK-18188
> Project: Spark
> Issue Type: Improvement
> Reporter: Davies Liu
> Assignee: Davies Liu
>
> There is an understanding issue for a long time:
> https://issues.apache.org/jira/browse/SPARK-4105, without any checksum for
> the blocks, it's very hard for us to identify where is the bug came from.
> Shuffle blocks are compressed separate (have checksum in it), but broadcast
> blocks are compressed together, we should add checksum for each of separately.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
