Re: log level in spark

That's all right, i manage to reduce the log level by removing the logback dependency in the pom.xml On Sat, May 11, 2019 at 02:54:49PM +0200, Nicolas Paris wrote: > Hi > > I have a spark code source with tests that create sparkSessions. > > I am running spark testing framework. > > My concer

Streaming job, catch exceptions

Hello, I am using Spark 2.2.1 with standalone resource manager. I have a streaming job where from time to time jobs are aborted due to the following exception. The reasons are different e.g. FileNotFound/NullPointerException etc org.apache.spark.SparkException: Job aborted due to stage failure: T

log level in spark

Hi I have a spark code source with tests that create sparkSessions. I am running spark testing framework. My concern is I am not able to configure the log level to INFO. I have large debug traces such: > DEBUG org.spark_project.jetty.util.Jetty - > java.lang.NumberFormatException: For input s