Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Sunita Arvind
te such code. >>> >>> >>> -- >>> *From:* Anthony May >>> *To:* Deepak Sharma ; Sunita Arvind < >>> sunitarv...@gmail.com> >>> *Cc:* "user@spark.apache.org" >>> *Sent:* Friday, May 6,

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Sunita Arvind
ctured Streaming; or if it > is, it's not obvious how to write such code. > > > -- > *From:* Anthony May > *To:* Deepak Sharma ; Sunita Arvind < > sunitarv...@gmail.com> > *Cc:* "user@spark.apache.org" > *Sent:* Friday, Ma

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Michael Armbrust
afka-fed >> Dataframe/Dataset that can be queried with Structured Streaming; or if it >> is, it's not obvious how to write such code. >> >> >> -- >> *From:* Anthony May >> *To:* Deepak Sharma ; Sunita Arvind < >> sunitarv...@gmail.com> >> *Cc

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Ted Yu
how to write such code. > > > -- > *From:* Anthony May > *To:* Deepak Sharma ; Sunita Arvind < > sunitarv...@gmail.com> > *Cc:* "user@spark.apache.org" > *Sent:* Friday, May 6, 2016 11:50 AM > *Subject:* Re: Adhoc queries on Sp

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Michael Malak
nd Cc: "user@spark.apache.org" Sent: Friday, May 6, 2016 11:50 AM Subject: Re: Adhoc queries on Spark 2.0 with Structured Streaming Yeah, there isn't even a RC yet and no documentation but you can work off the code base and test suites: https://github.com/apache/spark And this might h

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Anthony May
Yeah, there isn't even a RC yet and no documentation but you can work off the code base and test suites: https://github.com/apache/spark And this might help: https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/streaming/DataFrameReaderWriterSuite.scala On Fri,

Re: Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Deepak Sharma
Spark 2.0 is yet to come out for public release. I am waiting to get hands on it as well. Please do let me know if i can download source and build spark2.0 from github. Thanks Deepak On Fri, May 6, 2016 at 9:51 PM, Sunita Arvind wrote: > Hi All, > > We are evaluating a few real time streaming q

Adhoc queries on Spark 2.0 with Structured Streaming

2016-05-06 Thread Sunita Arvind
Hi All, We are evaluating a few real time streaming query engines and spark is my personal choice. The addition of adhoc queries is what is getting me further excited about it, however the talks I have heard so far only mention about it but do not provide details. I need to build a prototype to en