SOLVED! thanks for the help - I found the issue. it was the version of pyarrow (0.15.1) which apparently isn't currently stable. Downgrading it solved the issue for me
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
