Hi Tridib, For the second approach, can you attach the complete stack trace?
Thanks, Yin On Mon, Oct 20, 2014 at 8:24 PM, Michael Armbrust <mich...@databricks.com> wrote: > I think you are running into a bug that will be fixed by this PR: > https://github.com/apache/spark/pull/2850 > > On Mon, Oct 20, 2014 at 4:34 PM, tridib <tridib.sama...@live.com> wrote: > >> Hello Experts, >> After repeated attempt I am unable to run query on map json date string. I >> tried two approaches: >> >> *** Approach 1 *** created a Bean class with timespamp field. When I try >> to >> run it I get scala.MatchError: class java.sql.Timestamp (of class >> java.lang.Class). Here is the code: >> import java.sql.Timestamp; >> >> public class ComplexClaim { >> private Timestamp timestamp; >> >> public Timestamp getTimestamp() { >> return timestamp; >> } >> >> public void setTimestamp(Timestamp timestamp) { >> this.timestamp = timestamp; >> } >> } >> >> JavaSparkContext ctx = getCtx(sc); >> JavaSQLContext sqlCtx = getSqlCtx(getCtx(sc)); >> String path = "/hdd/spark/test.json"; >> JavaSchemaRDD test = sqlCtx.applySchema(ctx.textFile(path) , >> ComplexClaim.class); >> sqlCtx.registerRDDAsTable(test, "test"); >> execSql(sqlCtx, "select * from test", 1); >> >> >> >> *** Approach 2 *** >> Created a StructType to map the date field. I got scala.MatchError: >> TimestampType (of class >> org.apache.spark.sql.catalyst.types.TimestampType$). >> here is the code: >> >> public StructType createStructType() { >> List<StructField> fields = new ArrayList<StructField>(); >> fields.add(DataType.createStructField("timestamp", >> DataType.TimestampType, false)); >> return DataType.createStructType(fields); >> } >> >> public void testJsonStruct(SparkContext sc) { >> JavaSQLContext sqlCtx = getSqlCtx(getCtx(sc)); >> String path = "/hdd/spark/test.json"; >> JavaSchemaRDD test = sqlCtx.jsonFile(path, createStructType()); >> sqlCtx.registerRDDAsTable(test, "test"); >> execSql(sqlCtx, "select * from test", 1); >> } >> >> Input file has a single record: >> {"timestamp":"2014-10-10T01:01:01"} >> >> >> Thanks >> Tridib >> >> >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/spark-sql-timestamp-in-json-fails-tp16864.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
