Re: JdbcRDD Constructor

2015-10-20 Thread Deenar Toraskar
not have any such > column(unique values,numeric and sequential) which is suitable as Partition > Column to be specified for JDBCRDD Constructor or DataSource API.How to > proceed further on this scenario and also let me know if any default > approach Spark is going to implement if do

Re: JdbcRDD Constructor

2015-10-20 Thread satish chandra j
Hi Deenar, Thanks for your valuable inputs Here is a situation, if a Source Table does not have any such column(unique values,numeric and sequential) which is suitable as Partition Column to be specified for JDBCRDD Constructor or DataSource API.How to proceed further on this scenario and also

Re: JdbcRDD Constructor

2015-09-24 Thread Deenar Toraskar
t;>> >>>>> I am using Spark 1.5. I always get count = 100, irrespective of num >>>>> partitions. >>>>> >>>>> On Wed, Sep 23, 2015 at 5:00 PM, satish chandra j < >>>>> jsatishchan...@gmail.com> wrote: &

Re: JdbcRDD Constructor

2015-09-24 Thread satish chandra j
;> Satish Chandra >>>> >>>> On Wed, Sep 23, 2015 at 4:02 PM, Rishitesh Mishra < >>>> rishi80.mis...@gmail.com> wrote: >>>> >>>>> Which version of Spark you are using ?? I can get correct results >>>>> using JdbcR

Re: JdbcRDD Constructor

2015-09-23 Thread Deenar Toraskar
, Rishitesh Mishra < >>>> rishi80.mis...@gmail.com> wrote: >>>> >>>>> Which version of Spark you are using ?? I can get correct results >>>>> using JdbcRDD. Infact there is a test suite precisely for this ( >>>

Re: JdbcRDD Constructor

2015-09-23 Thread satish chandra j
e precisely for this (JdbcRDDSuite) >>> . >>> I changed according to your input and got correct results from this test >>> suite. >>> >>> On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < >>> jsatishchan...@gmail.com> wrote: >>

Re: JdbcRDD Constructor

2015-09-23 Thread Rishitesh Mishra
e) >> . >> I changed according to your input and got correct results from this test >> suite. >> >> On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < >> jsatishchan...@gmail.com> wrote: >> >>> HI All, >>> >>> JdbcRDD co

Re: JdbcRDD Constructor

2015-09-23 Thread satish chandra j
ing > JdbcRDD. Infact there is a test suite precisely for this (JdbcRDDSuite) . > I changed according to your input and got correct results from this test > suite. > > On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < > jsatishchan...@gmail.com> wrote: > >> HI All, >

Re: JdbcRDD Constructor

2015-09-23 Thread Rishitesh Mishra
All, > > JdbcRDD constructor has following parameters, > > *JdbcRDD > <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html#JdbcRDD(org.apache.spark.SparkContext,%20scala.Function0,%20java.lang.String,%20long,%20long,%20int,%20scala.Function1,%20s

JdbcRDD Constructor

2015-09-22 Thread satish chandra j
HI All, JdbcRDD constructor has following parameters, *JdbcRDD <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html#JdbcRDD(org.apache.spark.SparkContext, scala.Function0, java.lang.String, long, long, int, scala.Function1, scala.reflect.ClassTag)>*(SparkContext