Hi Ching-Mallete,
I have found the log and the reason for that.
Thanks a lot!Zhiliang
On Tuesday, November 10, 2015 12:23 PM, Zhiliang Zhu
wrote:
Also for Spark UI , that is, log from other places could be found, but the
log from the functions as mapPartitions could not.
Also for Spark UI , that is, log from other places could be found, but the log
from the functions as mapPartitions could not.
On Tuesday, November 10, 2015 11:52 AM, Zhiliang Zhu
wrote:
Dear Ching-Mallete ,
There are machines master01, master02 and master03 for the cluster, I cou
Dear Ching-Mallete ,
There are machines master01, master02 and master03 for the cluster, I could see
the stderr log and stdout log in the directories /.../yarn/logs under those
machines.In the stderr there are some notes, I just use log to follow the track
of the bug, however log in mapPartition
Hi Zhiliang,
You should be able to see them in the executor logs, which you can view via
the Spark UI, in the Executors page (stderr log).
HTH,
Deng
On Tue, Nov 10, 2015 at 11:33 AM, Zhiliang Zhu
wrote:
> Hi All,
>
> I need debug spark job, my general way is to print out the log, however,
> so
