[
https://issues.apache.org/jira/browse/SPARK-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-18140:
------------------------------
Summary: Parquet NPE / Update to 1.9 (was: Spark - parquet - null pointer
exception )
I think we might have to track this as "upgrade to Parquet 1.9" if anything. I
suspect it does atcually fix the issue but the userClassPathFirst settings
won't actually override the Parquet used in Spark.
> Parquet NPE / Update to 1.9
> ---------------------------
>
> Key: SPARK-18140
> URL: https://issues.apache.org/jira/browse/SPARK-18140
> Project: Spark
> Issue Type: Bug
> Reporter: dori waldman
>
> We are using spark to write parquet files ,
> We are getting nullPointerException due to this issue
> https://issues.apache.org/jira/browse/PARQUET-544
> trying to change to the new fixed parquet version (1.9.0) didnt worked
> In submit.sh added following parameters
> --jars "../lib/parquet-hadoop-1.9.0.jar"
> --conf "spark.driver.userClassPathFirst=true"
> --conf "spark.executor.userClassPathFirst=true"
> Any advise ? its in our production environment
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
