Re: Understanding how spark share db connections created on driver

2017-06-29 Thread Gourav Sengupta
Hi Sotiris, can you upload any sample data and then I will write the code and send it across to you? Do you have any particular used case for Influx DB? Regards, Gourav Sengupta On Thu, Jun 29, 2017 at 5:42 PM, Sotiris Beis wrote: > Hi Gourav, > > Do you have any suggestions of how the use o

Re: Understanding how spark share db connections created on driver

2017-06-29 Thread Sotiris Beis
Hi Gourav, Do you have any suggestions of how the use of dataframes will solve my problem? Cheers, Sotiris On Thu, 29 Jun 2017 at 17:37 Gourav Sengupta wrote: > Hi, > > I still do not understand why people do not use data frames. > > It makes you smile, take a sip of fine coffee, and feel good

Re: Understanding how spark share db connections created on driver

2017-06-29 Thread Gourav Sengupta
Hi, I still do not understand why people do not use data frames. It makes you smile, take a sip of fine coffee, and feel good about life and its all courtesy@SPARK. :) Regards, Gourav Sengupta On Thu, Jun 29, 2017 at 12:18 PM, Ryan wrote: > I think it creates a new connection on each worker,

Re: Understanding how spark share db connections created on driver

2017-06-29 Thread Ryan
I think it creates a new connection on each worker, whenever the Processor references Resource, it got initialized. There's no need for the driver connect to the db in this case. On Thu, Jun 29, 2017 at 5:52 PM, salvador wrote: > Hi all, > > I am writing a spark job from which at some point I wa