Hi, How can the DirectOutputCommitter be utilized for writing ORC files? I tried setting it via:
sc.getConf.set("spark.hadoop.mapred.output.committer.class","com.veracity-group.datastage.directorcwriter")
But I can still see a _temporary directory being used when I save my
dataframe as ORC.
Thank you,
Daniel
