Github user markgrover commented on the pull request:
https://github.com/apache/spark/pull/8007#issuecomment-132367263
>Actually @markgrover can you describe in more detail how you were trying
to use GetExecutorLossReason?
So, I uploaded my
[YARN](https://gist.github.com/markgrover/bb816fe8556e9498871a) and
[driver](https://gist.github.com/markgrover/3c4ef4e3a7823864bbd8) logs.
See
[here](https://gist.github.com/markgrover/bb816fe8556e9498871a#file-yarn-log-L92)
as to how the Executor Loss Reason is being asked for twice. I asserted that
it was only requested once by looking at the driver log. You can do so too by
searching for *Requesting loss reason for executorId: 2* in the [full driver
log](https://gist.githubusercontent.com/markgrover/3c4ef4e3a7823864bbd8/raw/ec8793874b8ff0545fd61f6bdc6e7f0681f9de1c/driver.log).
The relevant event receiving code snippet is
[here](https://github.com/markgrover/spark/blob/5a90c9926a396cf6b0b68ee3fabbfc67ae07dcf7/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala#L213)
and the relevant event sending code snippet is
[here](https://github.com/markgrover/spark/blob/5a90c9926a396cf6b0b68ee3fabbfc67ae07dcf7/core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala#L119).
This is all before merging your latest change since I was trying this out
last Friday. I wasn't able to figure out then why 2 events were being received.
Anyways, I realize this is a lot of info and perhaps, hard to go through. I
will poke at it more, perhaps, merge your latest commit and see if that helps.
I am envisioning some pain given that your and my pull requests are sharing
some code. If it continues for much longer, may be we should work off of the
same branch. I personally am indifferent to whether I rebase on yours, or
vice-versa.
I always also hoping to find a spark IRC channel where we could collaborate
real time but couldn't. I would definitely be open to hacking on this in a more
collaborative way if you think it'd help (I think it would).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
