Hi, sounds doable. I think it should be easy to set up a first proof of concept for this. Let us know if you need any further assistance.
Regards, Robert On Mon, May 30, 2016 at 2:29 PM, Palle <pa...@sport.dk> wrote: > Hi Robert. > > Thank you for the answer. > > I am looking at a rate of max 10.000 elements / 10 seconds, so > Elastic/Kibana is probably the way to go. I'll find a way to model it. > > Thanks. > > /Palle > > ----- Original meddelelse ----- > > *Fra:* Robert Metzger <rmetz...@apache.org> > *Til:* user@flink.apache.org <user@flink.apache.org> > *Dato:* Man, 30. maj 2016 12:31 > > *Emne:* Re: Visualize result of Flink job > > Hi Palle, > I think there is currently no way of sending the data from a streaming > Flink job into Zeppelin. > What rate / amount of data do you expect to send every 10 seconds to the > visualization tool? > People have used Flink -> ES -> Kibana for this purpose in the past [1], > but I think you can not send millions of records per second into ES. > Something like 1000 - 5000 elements / second should easily work for a small > ES setup. > [1] > https://www.elastic.co/blog/building-real-time-dashboard-applications-with-apache-flink-elasticsearch-and-kibana > > On Mon, May 30, 2016 at 10:00 AM, Palle <pa...@sport.dk> wrote: > >> OK, I found product that seems to be what I am looking for: Apache >> Zeppelin. I will have a look into that one. If anyone can point me to an >> example (Git) outputting data from Flink to the Zeppelin Notebook I would >> be happy. >> >> >> ----- Original meddelelse ----- >> > Fra: Palle <pa...@sport.dk> >> > Til: user@flink.apache.org >> > Dato: Man, 30. maj 2016 08:20 >> > Emne: Re: Visualize result of Flink job >> > >> > I know exactly what to visualize. As I wrote, it is the latest >> > result of the Flink job I would like to visualize. There is no need >> > to use elastic to find it first. >> > >> > The data I have is of such a nature that they every 10 seconds >> > could be written into a file, meaning that the file at all times >> > would contain the most recent results (at latest 10 seconds old). I >> > am not interested in the history, and therefore I should think >> > elastic is not the best fit. So my question is if anyone knows of a >> > component (Apache or other) that can make the visualization a little >> > nicer than just the file :-) >> > >> > >> > ----- Original meddelelse ----- >> > > Fra: Kanstantsin Kamkou <kkam...@gmail.com> >> > > Til: user@flink.apache.org >> > > Dato: Søn, 29. maj 2016 22:42 >> > > Emne: Re: Visualize result of Flink job >> > > >> > > > I am thinking it may not be the best fit, because Elastic is by >> > > nature a search engine that is good for trending and stuff like >> > that >> > > - not entire replacement of the current view. >> > > Why u think that the elasticsearch is not the right tool? To >> > > visualise >> > > something u have to find what to visualise first, right? >> > > >> > > >> > > On Sun, May 29, 2016 at 8:20 PM, Palle <pa...@sport.dk> wrote: >> > > > Hi there >> > > > >> > > > I am using Flink to analyse a lot of incoming data. Every 10 >> > > seconds it makes sense to present the analysis so far as some form >> > > of visualization. Every 10 seconds I therefore will replace the >> > > current contents of the visualization/presentation with the >> > analysis >> > > result of the most recent 10 seconds. >> > > > >> > > > I was first thinking of using ElasticSearch/Kibana for this >> > > because I know it should be easy to set up, but I am thinking it >> > may >> > > not be the best fit, because Elastic is by nature a search engine >> > > that is good for trending and stuff like that - not entire >> > > replacement of the current view. And therefore I may also >> > experience >> > > difficulties implementing the view in Elastic. >> > > > >> > > > Does anyone know of any other visualization tools that work well >> > > with Flink? ...where it is easy to export the result of a Flink >> > job >> > > to a user interface (web). >> > > > >> > > > Thanks >> > > > Palle >> > >> > > >
