Re: Reading table from sql database to apache spark dataframe/RDD

2017-05-01 Thread vincent gromakowski
Use cache or persist. The dataframe will be materialized when the 1st action is called and then be reused from memory for each following usage Le 1 mai 2017 4:51 PM, "Saulo Ricci" a écrit : > Hi, > > > I have the following code that is reading a table to a apache spark > DataFrame: > > val df =

Reading table from sql database to apache spark dataframe/RDD

2017-05-01 Thread Saulo Ricci
Hi, I have the following code that is reading a table to a apache spark DataFrame: val df = spark.read.format("jdbc") .option("url","jdbc:postgresql:host/database") .option("dbtable","tablename").option("user","username") .option("password", "password") .load() When I first