Convert timestamp to unix miliseconds

Hi All, I'm using spark 2.4 and trying to convert a timestamp column to unix with milliseconds using the unix_timestamp function. I tried to convert the result to double cast(unix_timestamp(timestamp) as double). I also tried using the timestamp format "-MM-dd HH:mm:ss.sss" and no matter what

Re: How can I write data to hive with jdbc

Actually if read from that table you can just do val tableName = "joinwarehouse.ods_job_log" option("dbtable", tableName) Also I made a typo in the url. val url = "jdbc:hive2://tidb4ser:11000/default" Just put default there for the database name. HTH view my Linkedin profile

Re: How can I write data to hive with jdbc

Caused by: org.apache.spark.sql.catalyst.analysis.NoSuchTableException: Table or view 'ods_job_log' not found in database 'default'; Where is your table? It is looking at *default database* In general this is the format read_df = spark.read. \ format("jdbc"). \

How can I write data to hive with jdbc

var cfg:Map[String,String] = Map() cfg += ("url"->"jdbc:hive2://tidb4ser:11000/joinwarehouse;user=jztwk;password=123456;hive.server2.proxy.user=jztwk") cfg += ("dbtable"->"ods_job_log") cfg += ("user"->"jztwk") cfg += ("passwrod"-> "123456") cfg += ("driver"-> "org.apache.h