you can out a try catch block in the map function and log the exception. The only "tricky" part is that the exception log will be located in the executor machine. Even if you don't do any trapping you should see the exception stacktrace in the executors' stderr log which is visible through the UI (if your app crashes the executor you can still see it as the last executor that ran on a given worker). But things like println and logging work inside map, you just have to remember everything happens on the remote machine
On Thu, Oct 16, 2014 at 8:11 PM, Ge, Yao (Y.) <y...@ford.com> wrote: > I need help to better trap Exception in the map functions. What is the > best way to catch the exception and provide some helpful diagnostic > information such as source of the input such as file name (and ideally line > number if I am processing a text file)? > > > > -Yao >
