Do you have a full stack trace? On Thu, Apr 2, 2015 at 11:45 AM, ogoh <oke...@gmail.com> wrote:
> > Hello, > My ETL uses sparksql to generate parquet files which are served through > Thriftserver using hive ql. > It especially defines a schema programmatically since the schema can be > only > known at runtime. > With spark 1.2.1, it worked fine (followed > > https://spark.apache.org/docs/latest/sql-programming-guide.html#programmatically-specifying-the-schema > ). > > I am trying to migrate into spark 1.3.0, but the API are confusing. > I am not sure if the example of > > https://spark.apache.org/docs/latest/sql-programming-guide.html#programmatically-specifying-the-schema > is still valid on Spark1.3.0? > For example, DataType.StringType is not there any more. > Instead, I found DataTypes.StringType etc. So, I migrated as below and it > builds fine. > But at runtime, it throws Exception. > > I appreciate any help. > Thanks, > Okehee > > == Exception thrown > java.lang.reflect.InvocationTargetException > scala.reflect.NameTransformer$.LOCAL_SUFFIX_STRING()Ljava/lang/String; > java.lang.NoSuchMethodError: > scala.reflect.NameTransformer$.LOCAL_SUFFIX_STRING()Ljava/lang/String; > > ==== my code's snippet > import org.apache.spark.sql.types.DataTypes; > DataTypes.createStructField(property, DataTypes.IntegerType, true) > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Generating-a-schema-in-Spark-1-3-failed-while-using-DataTypes-tp22362.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
