Re: In flight records on Flink : Newbie question

2021-10-06 Thread Declan Harrison
Many thanks Fabian for your prompt replies much appreciated Thanks Declan On Wed, Oct 6, 2021 at 8:38 AM Fabian Paul wrote: > Hi Declan, > > As far as I know the FileSink does not buffer records but writes the > records to temporary files which are bucketed later. For the Elasticsearch > sink >

Re: In flight records on Flink : Newbie question

2021-10-04 Thread Declan Harrison
Hi Fabian I am currently using the streaming file sink to local disk though potentially this sink could change to be Elastic Search. Declan On Mon, Oct 4, 2021 at 1:16 PM Fabian Paul wrote: > Hi Declan, > > I forgot to ask which sink you are using. I do not think it is generally > applicable t

HTTP or REST SQL Client

2021-10-02 Thread Declan Harrison
Hi All I know Flink has support for a command line SQL client however I was wondering if there is any support for providing a RESTful SQL client over HTTP for example? If not is this something the team would consider supporting in the near future? Thanks Declan

Re: In flight records on Flink : Newbie question

2021-10-01 Thread Declan Harrison
Hi Fabian Primarily more a case of understanding how many records are likely to be buffered by the sink still awaiting processing. So we are streaming event records to a sink for downstream processing in as close to real time as possible but wondered how many might be buffered by Flink and if that

In flight records on Flink : Newbie question

2021-09-30 Thread Declan Harrison
Hi Guys I've just recently started using Apache Flink to evaluate its suitability for a project I'm working on. First impressions are that the project is great, well documented and has lots of examples and guidance showcasing the multitude of things that it can do. Challenging knowing where to s