RE: Fwd: Writing to jdbc database from SparkR (1.5.2)

DataFrameWrite.jdbc() does not work? From: Felix Cheung [mailto:felixcheun...@hotmail.com] Sent: Sunday, February 7, 2016 9:54 AM To: Andrew Holway ; dev@spark.apache.org Subject: Re: Fwd: Writing to jdbc database from SparkR (1.5.2) Unfortunately I couldn't find a simple workaround. It seems to

pyspark worker concurrency

Hi, is it a good idea to have 2 threads in pyspark worker? - main thread responsible for receive and send data over socket while the other thread is calling user functions to process data? since CPU is idle (?) during network I/O, this should improve concurrency quite a bit. can expert answer t

Re: Fwd: Writing to jdbc database from SparkR (1.5.2)

Unfortunately I couldn't find a simple workaround. It seems to be an issue with DataFrameWriter.save() that does not work with jdbc source/format For instance, this does not work in Scala eitherdf1.write.format("jdbc").mode("overwrite").option("url", "jdbc:mysql://something.rds.amazonaws.com:330

Fwd: Writing to jdbc database from SparkR (1.5.2)

Hi, I have a thread on u...@spark.apache.org but I think this might require developer attention. I'm reading data from a database: This is working well. > df <- read.df(sqlContext, source="jdbc", url="jdbc:mysql:// database.foo.eu-west-1.rds.amazonaws.com:3306?user=user&password=pass