Thanks Marcelo! I will try to change the log4j.properties On Fri, Feb 20, 2015 at 11:37 AM, Marcelo Vanzin <[email protected]> wrote:
> Hi Anny, > > You could play with creating your own log4j.properties that will write > the output somewhere else (e.g. to some remote mount, or remote > syslog). Sorry, but I don't have an example handy. > > Alternatively, if you can use Yarn, it will collect all logs after the > job is finished and make them available as a single file using the > "yarn logs" command. > > > On Fri, Feb 20, 2015 at 11:31 AM, anny9699 <[email protected]> wrote: > > Hi, > > > > I am wondering if there's some way that could lead some of the worker > stdout > > to one place instead of in each worker's stdout. For example, I have the > > following code > > > > RDD.foreach{line => > > try{ > > do something > > }catch{ > > case e:exception => println(line) > > } > > } > > > > Every time I want to check what's causing the exception, I have to check > one > > worker after another in the UI, because I don't know which worker will be > > dealing with the exception case. Is there a way that the "println" could > > print to one place instead of separate worker stdout so that I only need > to > > check one place? > > > > Thanks a lot! > > Anny > > > > > > > > -- > > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/output-worker-stdout-to-one-place-tp21742.html > > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > -- > Marcelo >
