Update:
I tried surrounding the problematic code with try and catch but that does
not do the trick:
try
{
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext._
val jsonFiles=sqlContext.jsonFile("/requests.loading")
} catch {
case _: Throwable => // Catching all exceptions an
Hi,
I'm loading a bunch of json files and there seems to be problems with
specific files (either schema changes or incomplete files).
I'd like to catch the inconsistent files but I'm not sure how to do it.
This is the exception I get:
14/11/20 00:13:49 INFO cluster.YarnClientClusterScheduler: Remo
