Hi All,
I am using spark version - 1.6.1
I have a text table in hive having `timestamp` datatype with nanoseconds
precision.
Hive Table Schema:
c_timestamp timestamp
Hive Table data:
hive> select * from tbl1;
OK
00:00:00.000000001
12:12:12.123456789
23:59:59.999999999
But as per the docs, from Spark 1.5
Timestamps are now stored at a precision of 1us, rather than 1ns
Sample code:
SparkConf conf = new
SparkConf(true).setMaster("yarn-cluster").setAppName("SAMPLE_APP");
SparkContext sc = new SparkContext(conf);
HiveContext hc = new HiveContext(sc);
DataFrame df = hc.table("testdb.tbl1");
Data is truncated to microseconds.
00:00:00
12:12:12.123456
23:59:59.999999
Is there any way to use nanoseconds here?
Regards,
Devender
________________________________
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
