RE: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Mohammed Guller
08 AM To: Mohammed Guller Cc: Kelvin Chu; user@spark.apache.org Subject: Re: using a database connection pool to write data into an RDBMS from a Spark application Hm, others can correct me if I'm wrong, but is this what SPARK_CLASSPATH is for? On Fri, Feb 20, 2015 at 6:04 PM, Mohammed Gul

Re: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Sean Owen
oad >> BoneCP 0.8.0? I tried the --jars and --driver-classpath flags, but it didn't >> help. >> >> Thanks, >> Mohammed >> >> >> -Original Message- >> From: Sean Owen [mailto:so...@cloudera.com] >> Sent: Friday, February 20, 2015

RE: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Mohammed Guller
...@cloudera.com] Sent: Friday, February 20, 2015 9:42 AM To: Mohammed Guller Cc: Kelvin Chu; user@spark.apache.org Subject: Re: using a database connection pool to write data into an RDBMS from a Spark application Have a look at spark.yarn.user.classpath.first and spark.files.userClassPathFirst for a

Re: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Sean Owen
Friday, February 20, 2015 2:06 AM > To: Mohammed Guller > Cc: Kelvin Chu; user@spark.apache.org > Subject: Re: using a database connection pool to write data into an RDBMS > from a Spark application > > Although I don't know if it's related, the Class.forName() metho

RE: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Mohammed Guller
2015 2:06 AM To: Mohammed Guller Cc: Kelvin Chu; user@spark.apache.org Subject: Re: using a database connection pool to write data into an RDBMS from a Spark application Although I don't know if it's related, the Class.forName() method of loading drivers is very old. You should be using DataS

Re: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-20 Thread Sean Owen
...@gmail.com] > Sent: Thursday, February 19, 2015 7:56 PM > To: Mohammed Guller > Cc: user@spark.apache.org > Subject: Re: using a database connection pool to write data into an RDBMS > from a Spark application > > > > Hi Mohammed, > > > > Did you use --jars to

RE: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-19 Thread Mohammed Guller
Guller Cc: user@spark.apache.org Subject: Re: using a database connection pool to write data into an RDBMS from a Spark application Hi Mohammed, Did you use --jars to specify your jdbc driver when you submitted your job? Take a look of this link: http://spark.apache.org/docs/1.2.0/submitting

Re: using a database connection pool to write data into an RDBMS from a Spark application

2015-02-19 Thread Kelvin Chu
Hi Mohammed, Did you use --jars to specify your jdbc driver when you submitted your job? Take a look of this link: http://spark.apache.org/docs/1.2.0/submitting-applications.html Hope this help! Kelvin On Thu, Feb 19, 2015 at 7:24 PM, Mohammed Guller wrote: > Hi – > > I am trying to use Bone

using a database connection pool to write data into an RDBMS from a Spark application

2015-02-19 Thread Mohammed Guller
Hi – I am trying to use BoneCP (a database connection pooling library) to write data from my Spark application to an RDBMS. The database inserts are inside a foreachPartition code block. I am getting this exception when the code tries to insert data using BoneCP: java.sql.SQLException: No suita