Question on using pseudo columns in spark jdbc options

2017-12-02 Thread रविशंकर नायर
Hi all, I am using a query to fetch data from MYSQL as follows: var df = spark.read. format("jdbc"). option("url", "jdbc:mysql://10.0.0.192:3306/retail_db"). option("driver" ,"com.mysql.jdbc.Driver"). option("user", "retail_dba"). option("password", "cloudera"). option("dbtable", "orders"). optio

Re: Is Databricks REST API open source ?

2017-12-02 Thread kant kodali
got it! Thanks! On Sat, Dec 2, 2017 at 9:46 PM, Holden Karau wrote: > That API is not open source. There are some other options as separate > projects you can check out (like Livy,spark-jobserver, etc). > > On Sat, Dec 2, 2017 at 8:30 PM kant kodali wrote: > >> HI All, >> >> Is REST API (https:

Re: Is Databricks REST API open source ?

2017-12-02 Thread Holden Karau
That API is not open source. There are some other options as separate projects you can check out (like Livy,spark-jobserver, etc). On Sat, Dec 2, 2017 at 8:30 PM kant kodali wrote: > HI All, > > Is REST API (https://docs.databricks.com/api/index.html) open source? > where I can submit spark jobs

Recommended way to serialize Hadoop Writables' in Spark

2017-12-02 Thread pradeepbaji
Hi, Is there any recommended way of serializing Hadoop Writables' in Spark? Here is my problem. Question1: I have a pair RDD which is created by reading a SEQ[LongWritable, BytesWritable]: RDD[(LongWritable, BytesWritable)] I have these two settings set in spark conf. spark.serializer=org.apach

Is Databricks REST API open source ?

2017-12-02 Thread kant kodali
HI All, Is REST API (https://docs.databricks.com/api/index.html) open source? where I can submit spark jobs over rest? Thanks!