Re: What's the best practice to load data from RDMS to Spark

2017-01-02 Thread Jacek Laskowski
FYI option works with boolean literals directly. Jacek On 30 Dec 2016 9:32 p.m., "Palash Gupta" wrote: > Hi, > > If you want to load from csv, you can use below procedure. Of course you > need to define spark context first. (Given example to load all csv under a > folder, you can use specific n

Re: What's the best practice to load data from RDMS to Spark

2016-12-30 Thread Palash Gupta
Hi, If you want to load from csv, you can use below procedure. Of course you need to define spark context first. (Given example to load all csv under a folder, you can use specific name for single file) // these lines are equivalent in Spark 2.0 spark.read.format("csv").option("header", "true").l

What's the best practice to load data from RDMS to Spark

2016-12-30 Thread Raymond Xie
Hello, I am new to Spark, as a SQL developer, I only took some courses online and spent some time myself, never had a chance working on a real project. I wonder what would be the best practice (tool, procedure...) to load data (csv, excel) into Spark platform? Thank you. *Raymond*