SnappyData should work well for what you want, it deeply integrates an in-memory database with Spark which supports ingesting streaming data and concurrently querying it from a dashboard. SnappyData currently has an integration with Apache Zeppelin (notebook visualization) and soon it will have one with Tableau. Finally, the database and Spark executors share the same JVM in SnappyData, so you will get better performance than any database that works with Spark over a connector.
Repo: https://github.com/SnappyDataInc/snappydata Using it with Zeppelin: https://github.com/SnappyDataInc/snappydata/blob/master/docs/aqp_aws.md#using-apache-zeppelin Hope this helps On Thu, Mar 30, 2017 at 7:56 AM, Alonso Isidoro Roman <alons...@gmail.com> wrote: > you can check if you want this link > <https://github.com/alonsoir/twitter-sentiment-analysis> > > > elastic, kibana and spark working together. > > Alonso Isidoro Roman > [image: https://]about.me/alonso.isidoro.roman > > <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> > > 2017-03-30 16:25 GMT+02:00 Miel Hostens <miel.host...@ugent.be>: > >> We're doing exactly same thing over here! >> >> Spark + ELK >> >> Met vriendelijk groeten, >> >> *Miel Hostens*, >> >> *B**uitenpraktijk+* >> Department of Reproduction, Obstetrics and Herd Health >> <http://www.rohh.ugent.be/> >> >> Ambulatory Clinic >> Faculty of Veterinary Medicine <http://www.ugent.be/di> >> Ghent University <http://www.ugent.be/> >> Salisburylaan 133 >> 9820 Merelbeke >> Belgium >> >> >> >> Tel: +32 9 264 75 28 <+32%209%20264%2075%2028> >> >> Fax: +32 9 264 77 97 <+32%209%20264%2077%2097> >> >> Mob: +32 478 59 37 03 <+32%20478%2059%2037%2003> >> >> e-mail: miel.host...@ugent.be >> >> On 30 March 2017 at 15:01, Szuromi Tamás <trom...@gmail.com> wrote: >> >>> For us, after some Spark Streaming transformation, Elasticsearch + >>> Kibana is a great combination to store and visualize data. >>> An alternative solution that we use is Spark Streaming put some data >>> back to Kafka and we consume it with nodejs. >>> >>> Cheers, >>> Tamas >>> >>> 2017-03-30 9:25 GMT+02:00 Alonso Isidoro Roman <alons...@gmail.com>: >>> >>>> Read this first: >>>> >>>> http://www.oreilly.com/data/free/big-data-analytics-emerging >>>> -architecture.csp >>>> >>>> https://www.ijircce.com/upload/2015/august/97_A%20Study.pdf >>>> >>>> http://www.pentaho.com/assets/pdf/CqPxTROXtCpfoLrUi4Bj.pdf >>>> >>>> http://www.gartner.com/smarterwithgartner/six-best-practices >>>> -for-real-time-analytics/ >>>> >>>> https://speakerdeck.com/elasticsearch/using-elasticsearch-lo >>>> gstash-and-kibana-to-create-realtime-dashboards >>>> >>>> https://www.youtube.com/watch?v=PuvHINcU9DI >>>> >>>> then take a look to >>>> >>>> https://kudu.apache.org/ >>>> >>>> Tell us later what you think. >>>> >>>> >>>> >>>> >>>> Alonso Isidoro Roman >>>> [image: https://]about.me/alonso.isidoro.roman >>>> >>>> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >>>> >>>> 2017-03-30 7:14 GMT+02:00 Gaurav Pandya <gauravhpan...@gmail.com>: >>>> >>>>> Hi Noorul, >>>>> >>>>> Thanks for the reply. >>>>> But then how to build the dashboard report? Don't we need to store >>>>> data anywhere? >>>>> Please suggest. >>>>> >>>>> Thanks. >>>>> Gaurav >>>>> >>>>> On Thu, Mar 30, 2017 at 10:32 AM, Noorul Islam Kamal Malmiyoda < >>>>> noo...@noorul.com> wrote: >>>>> >>>>>> I think better place would be a in memory cache for real time. >>>>>> >>>>>> Regards, >>>>>> Noorul >>>>>> >>>>>> On Thu, Mar 30, 2017 at 10:31 AM, Gaurav1809 <gauravhpan...@gmail.com> >>>>>> wrote: >>>>>> > I am getting streaming data and want to show them onto dashboards >>>>>> in real >>>>>> > time? >>>>>> > May I know how best we can handle these streaming data? where to >>>>>> store? (DB >>>>>> > or HDFS or ???) >>>>>> > I want to give users a real time analytics experience. >>>>>> > >>>>>> > Please suggest possible ways. Thanks. >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > View this message in context: http://apache-spark-user-list. >>>>>> 1001560.n3.nabble.com/How-best-we-can-store-streaming-data-o >>>>>> n-dashboards-for-real-time-user-experience-tp28548.html >>>>>> > Sent from the Apache Spark User List mailing list archive at >>>>>> Nabble.com. >>>>>> > >>>>>> > ------------------------------------------------------------ >>>>>> --------- >>>>>> > To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>>>>> > >>>>>> >>>>> >>>>> >>>> >>> >> >
