Re: Data from PostgreSQL to Spark

2015-08-03 Thread Jeetendra Gangele
;>> Parallel i will be hitting around 6000 call to postgreSQl which is not >>>>> good my database will die. >>>>> these calls to database will keeps on increasing. >>>>> Handling millions on request is not an issue with Hbase/NOSQL >>>>&g

Re: Data from PostgreSQL to Spark

2015-07-28 Thread Jörn Franke
eeps on increasing. >>>> Handling millions on request is not an issue with Hbase/NOSQL >>>> >>>> any other alternative? >>>> >>>> >>>> >>>> >>>> On 27 July 2015 at 23:18, wrote: >>>> >>>

Re: Data from PostgreSQL to Spark

2015-07-28 Thread Jeetendra Gangele
>>> >>> any other alternative? >>> >>> >>> >>> >>> On 27 July 2015 at 23:18, wrote: >>> >>>> You can have Spark reading from PostgreSQL through the data access API. >>>> Do you have any concern with that

Re: Data from PostgreSQL to Spark

2015-07-28 Thread santoshv98
5 at 23:18, wrote: You can have Spark reading from PostgreSQL through the data access API. Do you have any concern with that approach since you mention copying that data into HBase. From: Jeetendra Gangele Sent: Monday, July 27, 6:00 AM Subject: Data from PostgreSQL to Spark To: user Hi All

Re: Data from PostgreSQL to Spark

2015-07-28 Thread Jeetendra Gangele
an issue with Hbase/NOSQL >> >> any other alternative? >> >> >> >> >> On 27 July 2015 at 23:18, wrote: >> >>> You can have Spark reading from PostgreSQL through the data access API. >>> Do you have any concern with that approach s

Re: Data from PostgreSQL to Spark

2015-07-28 Thread Jeetendra Gangele
;> any other alternative? >> >> >> >> >> On 27 July 2015 at 23:18, wrote: >> >>> You can have Spark reading from PostgreSQL through the data access API. >>> Do you have any concern with that approach since you mention copying that >>>

Re: Data from PostgreSQL to Spark

2015-07-27 Thread santoshv98
Monday, July 27, 6:00 AM Subject: Data from PostgreSQL to Spark To: user Hi All I have a use case where where I am consuming the Events from RabbitMQ using spark streaming.This event has some fields on which I want to query the PostgreSQL and bring the data and then do the join between

Re: Data from PostgreSQL to Spark

2015-07-27 Thread santoshv98
have Spark reading from PostgreSQL through the data access API. Do you have any concern with that approach since you mention copying that data into HBase. From: Jeetendra Gangele Sent: Monday, July 27, 6:00 AM Subject: Data from PostgreSQL to Spark To: user Hi All I have a use case whe

Re: Data from PostgreSQL to Spark

2015-07-27 Thread ayan guha
; > > > On 27 July 2015 at 23:18, wrote: > >> You can have Spark reading from PostgreSQL through the data access API. >> Do you have any concern with that approach since you mention copying that >> data into HBase. >> >> From: Jeetendra Gangele >> Sent: Mon

Re: Data from PostgreSQL to Spark

2015-07-27 Thread Jeetendra Gangele
: > You can have Spark reading from PostgreSQL through the data access API. Do > you have any concern with that approach since you mention copying that data > into HBase. > > From: Jeetendra Gangele > Sent: Monday, July 27, 6:00 AM > Subject: Data from PostgreSQL to Spark > T

Re: Data from PostgreSQL to Spark

2015-07-27 Thread felixcheung_m
You can have Spark reading from PostgreSQL through the data access API. Do you have any concern with that approach since you mention copying that data into HBase. From: Jeetendra Gangele Sent: Monday, July 27, 6:00 AM Subject: Data from PostgreSQL to Spark To: user Hi All I have a

Data from PostgreSQL to Spark

2015-07-27 Thread Jeetendra Gangele
Hi All I have a use case where where I am consuming the Events from RabbitMQ using spark streaming.This event has some fields on which I want to query the PostgreSQL and bring the data and then do the join between event data and PostgreSQl data and put the aggregated data into HDFS, so that I run