Re: Does Zeppelin 0.7.1 work with Spark Structured Streaming 2.1.1?

2017-05-19 Thread kant kodali
Look for something that is more like in this video where the graphs automatically update themselves. Is that possible in Zeppelin? https://www.youtube.com/watch?v=IJmFTXvUZgY You can watch it from 9:20 On Fri, May 19, 2017 at 9:21 AM, kant kodali wrote: > Hi All, > > I have the

Does Zeppelin 0.7.1 work with Spark Structured Streaming 2.1.1?

2017-05-19 Thread kant kodali
Hi All, I have the following code StreamingQuery query = df2.writeStream().outputMode("complete").queryName( "foo").option("truncate", "false").format("console").start(); query.awaitTermination(); and it works fine however when I change it to the below code. I do get the output but only once a

How to print data to console in structured streaming using Spark 2.1.0 and Zeppelin 0.7.1?

2017-05-16 Thread kant kodali
Hi All, I have the following code. val ds = sparkSession.readStream() .format("kafka") .option("kafka.bootstrap.servers",bootstrapServers)) .option("subscribe", topicName) .option("checkpointLocation", hdfsCheckPointDir)

Re: How to create a real time dashboards from spark using web socket?

2017-04-25 Thread kant kodali
Cheung wrote: > You could have multiple notes sharing interpreter - could you have one > note for setup to call streamingcontext.start and another note on a > schedule just to run the select sql statement? > > _____ > From: kant kodali > Sent: Tuesday

Re: How to create a real time dashboards from spark using web socket?

2017-04-25 Thread kant kodali
uyHai Doan wrote: > Yes, the scheduler applies to the entire note, not just paragraph > Le 25 avr. 2017 02:39, "kant kodali" a écrit : > >> Also it doesn't look like we can run a scheduler on one paragraph? we >> have to run the scheduler for the entire noteboo

Re: How to create a real time dashboards from spark using web socket?

2017-04-24 Thread kant kodali
Also it doesn't look like we can run a scheduler on one paragraph? we have to run the scheduler for the entire notebook always? On Mon, Apr 24, 2017 at 1:06 PM, kant kodali wrote: > Hi All, > > I currently do the following > > val jsonDStream = getJsonDStream() > >

How to create a real time dashboards from spark using web socket?

2017-04-24 Thread kant kodali
Hi All, I currently do the following val jsonDStream = getJsonDStream() jsonDStream.foreachRDD{rdd => val jsonDF = spark.read.json(rdd) jsonDF.createOrReplaceTempView("dataframe") } client.startStream() %spark.sql select * from dataframe I can see the data and everytime I click a run b

Re: Zeppelin Spark Streaming Twitter Stuck

2017-04-24 Thread kant kodali
@Chaoran Yu Yeah I don't think its dependency issue. you wouldn't be able to call methods if you are missing dependencies. I am also in a similar boat though I am trying to get Streaming and Zeppelin to work except I have my own indirect receiver (not the direct stream). That twitter example is pr

Re: Zeppelin Spark Streaming Twitter Stuck

2017-04-24 Thread kant kodali
ndf On Sun, Apr 23, 2017 at 12:11 PM, kant kodali wrote: > @Chaoran Yu Yeah I don't think its dependency issue. you wouldn't be able > to call methods if you are missing dependencies. > > I am also in a similar boat though I am trying to get Streaming and > Zeppelin

Re: java.lang.NullpointerException

2017-04-22 Thread kant kodali
FYI: I am using Spark Standalone mode On Sat, Apr 22, 2017 at 10:57 PM, kant kodali wrote: > Hi All, > > I get the below stack trace when I am using Zeppelin. If I don't use > Zeppelin all my client Jobs are running fine. I am using spark 2.1.0 > > I am not sure why Zeppe

java.lang.NullpointerException

2017-04-22 Thread kant kodali
Hi All, I get the below stack trace when I am using Zeppelin. If I don't use Zeppelin all my client Jobs are running fine. I am using spark 2.1.0 I am not sure why Zeppelin is unable to create a SparkContext ? but then it says it created SparkSession so it doesn't seem to make a lot of sense. any

Re: How to use web sockets with Zeppellin

2016-09-08 Thread kant kodali
@Corneau The links you have pointed out doesnt show how I can retrieve messages through websocket and tie to chars. I would stress again and say how do i tie all this up to charts? On Thu, Sep 8, 2016 at 12:17 AM, kant kodali wrote: > Hi Corneau, > > I appreciate the responses but a

Re: How to use web sockets with Zeppellin

2016-09-08 Thread kant kodali
ou will find the list of existing interpreters in the menu. > > If you do need to create one, then you can refer to this guide > https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/development/ > writingzeppelininterpreter.html > > On Thu, Sep 8, 2016 at 3:56 PM, kant kodali wrote: > >>

Re: How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
ssage from backend, a frontend event handler may also > required. > > > 在 2016年9月8日,11:39,kant kodali 写道: > > ok here is a concrete example. I am still trying to connect the dots..If I > were to split Zeppelin as both front end and backend then I wouldn't choose > to us

Re: How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
://www.zeppelinhub.com/viewer/showcases/Streaming > > On Thu, Sep 8, 2016 at 3:30 AM, kant kodali wrote: > >> Hi! Thanks for this but I still see the big picture. I have not used >> Angular before nor I am a frontend guy. so I have the following questions. >> >> &

Re: How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
ep 7, 2016 5:05 AM, spacewalk...@163.com spacewalk...@163.com wrote: @kant kodali change   zeppelin-web\src\components\baseUrl\baseUrl.service.js like this: this.getWebsocketUrl = function() {    var wsProtocol = location.protocol === 'https:' ? 'wss:' : 'ws:';  

How to use web sockets with Zeppellin

2016-09-07 Thread kant kodali
HI Guys, I have a server which can handle websocket connections. Now I would like to see if there is a way to tell apache zeppellin to create to websocket connect to my server and display the whatever it received? Thanks, Kant

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
n Thu, Aug 25, 2016 at 11:54 AM, kant kodali < kanth...@gmail.com > wrote: yeah In a real time dashboard we need less than a second latency or even millisecond. On Wed, Aug 24, 2016 7:42 PM, Vinay Shukla vinayshu...@gmail.com wrote: Kant, Moon referred to the attached feature. You can have

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
Aug 25, 2016 at 11:19 AM, kant kodali < kanth...@gmail.com > wrote: I dont know how angular works moreover I don't understand why we should be tied to certain framework? On Wed, Aug 24, 2016 6:53 PM, Corneau Damien cornead...@gmail.com wrote: Wouldn't angular bind work in that case

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
> wrote: Try find small 'clock' icon on any particular Note next to 'Remove' button. This scheduler feature will help you auto refresh Note. Thanks, moon On Wed, Aug 24, 2016 at 11:31 AM kant kodali < kanth...@gmail.com > wrote: at very least can we auto refreshing o

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
at very least can we auto refreshing on the Zeppelin dashboard on a certain time interval? On Wed, Aug 24, 2016 10:56 AM, kant kodali kanth...@gmail.com wrote: I would say this is like a very basic requirement for a dashboard and it is surprising Zeppelin doesn't have an out of th

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
he data source so that the new updates can be pushed directly to the paragraph On Wed, Aug 24, 2016 at 1:15 PM, kant kodali < kanth...@gmail.com > wrote: Hi, Sure, I can either write it to HDFS or Cassandra but not HBASE since we don't use it but how about constant updates? you kno

Re: is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
ts. Thanks, Vinay On Wednesday, August 24, 2016, kant kodali < kanth...@gmail.com > wrote: Hi Guys, is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and wr

is there way to update(push latest data to) Zeppelin dashboard ?

2016-08-24 Thread kant kodali
Hi Guys, is there way to update(push latest data to but not pull) Zeppelin dashboard ? for example I get bunch of data to my spark streaming cluster and as I get more and more data I want to compute and write it somewhere such that my Zeppelin dashboard is updated? If so, where would be the best p