[
https://issues.apache.org/jira/browse/SPARK-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838681#comment-15838681
]
Sven Krasser commented on SPARK-4049:
-------------------------------------
[~srowen], see my comment from before:
{quote}
As a user, when I see 100% on the UI, then I assume the RDD is fully cached.
However, this could also mean that some partitions are missing (and need to be
recomputed) and some are overreplicated.
{quote}
In other words, I think mixing replication and cached fraction adds confusion.
If this must be one number, then it can't be communicated easily that an RDD is
fully cached and doesn't require any recomputations. When diagnosing job
problems, knowing that an RDD is fully cached is more important than knowing
that it is cached redundantly.
> Storage web UI "fraction cached" shows as > 100%
> ------------------------------------------------
>
> Key: SPARK-4049
> URL: https://issues.apache.org/jira/browse/SPARK-4049
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.2.0
> Reporter: Josh Rosen
> Priority: Minor
>
> In the Storage tab of the Spark Web UI, I saw a case where the "Fraction
> Cached" was greater than 100%:
> !http://i.imgur.com/Gm2hEeL.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
