Awesome ! Thanks a lot ! I should probably write a blog post somewhere explaining this.
On Tue, Jul 5, 2016 at 4:28 PM, Chesnay Schepler <ches...@apache.org> wrote: > They serve a similar purpose. > > OutputFormats originate from the Batch API, whereas SinkFunctions are a > Streaming API concept. > > You can however use OutputFormats in the Streaming API using the > DataStrea#writeUsingOutputFormat. > > Regards, > Chesnay > > > On 05.07.2016 12:51, Harikrishnan S wrote: > > Ah that makes send. Also what's the difference between a RichOutputFormat > and a RichSinkFunction ? Can I use JDBCOutputFormat as a sink in a stream ? > > On Tue, Jul 5, 2016 at 3:53 PM, Chesnay Schepler <ches...@apache.org> > wrote: > >> Hello, >> >> an instance of the JDBCOutputFormat will use a single connection to send >> all values. >> >> Essentially >> - open(...) is called at the very start to create the connection >> - then all invoke/writeRecord calls are executed (using the same >> connection) >> - then close() is called to clean up. >> >> The total number of connections made to the database depends on the >> parallelism of the Sink, as every parallel instance creates it's own >> connection. >> >> Regards, >> Chesnay >> >> >> On 05.07.2016 12:04, Harikrishnan S wrote: >> >> The basic idea was that I would create a pool of connections in the >> open() method in a custom sink and each invoke() method gets one connection >> from the pool and does the upserts needed. I might have misunderstood how >> sinks work in flink though. >> >> On Tue, Jul 5, 2016 at 2:22 PM, Flavio Pompermaier < >> <pomperma...@okkam.it>pomperma...@okkam.it> wrote: >> >>> why do you need a connection pool? >>> On 5 Jul 2016 11:41, "Harikrishnan S" < <hihari...@gmail.com> >>> hihari...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> Are there any examples of implementing a jdbc sink in flink using a >>>> connection pool ? >>>> >>>> Thanks >>>> >>>> On Tue, Jul 5, 2016 at 2:00 PM, Harikrishnan S < <hihari...@gmail.com> >>>> hihari...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Are there any examples of implementing a jdbc sink in flink using a >>>>> connection pool ? >>>>> >>>>> Thanks >>>>> >>>>> On Tue, Jul 5, 2016 at 1:57 PM, Harikrishnan S < <hihari...@gmail.com> >>>>> hihari...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Are there any examples of implementing a jdbc sink in flink using a >>>>>> connection pool ? >>>>>> >>>>>> Thanks >>>>>> >>>>> >>>>> >>>> >> >> > >
