I have this ORC file that was generated by a Spark 1.6 program. It opens fine in Spark 1.6 with 6GB of driver memory, and probably less.
However, when I try to open the same file in Spark 2.0 or 2.1, I get GC timeout exceptions. And this is with 6, 8, and even 10GB of driver memory. This is strange and smells like buggy behavior. How can I debug this or workaround it in Spark 2+? Nick -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Reading-ORC-file-fine-on-1-6-GC-timeout-on-2-tp28654.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
