[
https://issues.apache.org/jira/browse/SPARK-19437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shixiong Zhu resolved SPARK-19437.
----------------------------------
Resolution: Fixed
Assignee: jin xing
Fix Version/s: 2.2.0
> ExecutorId in HearbeatReceiverSuite is incorrect.
> -------------------------------------------------
>
> Key: SPARK-19437
> URL: https://issues.apache.org/jira/browse/SPARK-19437
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: jin xing
> Assignee: jin xing
> Fix For: 2.2.0
>
>
> The current code in *HeartbeatReceiverSuite*, executorId is set as below:
> {code}
> private val executorId1 = "executor-1"
> private val executorId2 = "executor-2"
> {code}
> The executorId is sent to driver when register as below:
> {code}
> test("expire dead hosts should kill executors with replacement (SPARK-8119)")
> {
> ...
> fakeSchedulerBackend.driverEndpoint.askSync[Boolean](
> RegisterExecutor(executorId1, dummyExecutorEndpointRef1, "1.2.3.4", 0,
> Map.empty))
> ...
> }
> {code}
> Receiving *RegisterExecutor*, the executorId will be compared with
> *currentExecutorIdCounter* as below:
> {code}
> case RegisterExecutor(executorId, executorRef, hostname, cores, logUrls) =>
> if (executorDataMap.contains(executorId)) {
> executorRef.send(RegisterExecutorFailed("Duplicate executor ID: " +
> executorId))
> context.reply(true)
> } else {
> ...
> executorDataMap.put(executorId, data)
> if (currentExecutorIdCounter < executorId.toInt) {
> currentExecutorIdCounter = executorId.toInt
> }
> ...
> {code}
> *executorId.toInt* will cause NumberformatException.
> This unit test can pass currently because of *askWithRetry*, when catching
> exception, RPC will call again, thus it will go *if* branch and return true.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
