I've tried starting a Log Server on the driver according with this doc https://docs.python.org/2/howto/logging-cookbook.html
so the workers could connect and send their logs to this remote server. If I remember correctly ... I didn't get any error messages and the job worked fine, but ... the logs were missing ... (I wasn't able to find them anywhere, at least). . 2014-11-21 1:35 GMT+01:00 freedafeng [via Apache Spark User List] < ml-node+s1001560n19432...@n3.nabble.com>: > Hi, > > I am wondering how to write logging info in a worker when running a > pyspark app. I saw the thread > > > http://apache-spark-user-list.1001560.n3.nabble.com/logging-in-pyspark-td5458.html > > but did not see a solution. Anybody know a solution? Thanks! > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/logging-in-workers-for-pyspark-tp19432.html > To start a new topic under Apache Spark User List, email > ml-node+s1001560n1...@n3.nabble.com > To unsubscribe from Apache Spark User List, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=YW5nZWwuYWx2YXJlei5wYXNjdWFAZ21haWwuY29tfDF8ODAzOTc5ODky> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/logging-in-workers-for-pyspark-tp19432p19445.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
