Hi
I have a process that writes to multiple partitions of the same table in
parallel using multiple threads sharing the same SQL context
df.write.partitionedBy("partCol").insertInto("tableName") . I am
getting FileNotFoundException on _temporary directory. Each write only goes
to a single partition, is there some way of not using dynamic partitioning
using df.write without having to resort to .save as I dont want to hardcode
a physical DFS location in my code?
This is similar to this issue listed here
https://issues.apache.org/jira/browse/SPARK-2984
Regards
Deenar
*Think Reactive Ltd*
deenar.toras...@thinkreactive.co.uk
