We will look into streaming-streaming joins in future release of Spark, though no promises on any timeline yet. We are currently fighting to get Spark 2.0 out of the door. There isnt a JIRA for this right now. However, you can track the Structured Streaming Epic JIRA to track whats going on. I try to post major feature stuff as sub jiras of this master jira. https://issues.apache.org/jira/browse/SPARK-8360
On Wed, Jul 6, 2016 at 8:19 PM, craigjar <craig.jarc...@gmail.com> wrote: > I have been doing several Spark PoC projects recently and the latest one > involved the new 2.0 experimental feature Structured Streaming. My PoC > ended up being a non-starter as I quickly realized the stream to stream > joins are not implemented yet. I believe this feature will be immensely > powerful and allow applications to migrate from batch to streaming with > ease, which is exactly what I would like to accomplish. > > I came across an interesting article (link below) from another streaming > platform called AMPS which I believe describes a common use case where > Spark's Structured Streaming would be perfect for and I hope the Spark dev > team is looking at the AMPS platform for some inspiration. > > > http://www.crankuptheamps.com/blog/posts/2014/04/07/real-time-streaming-joins-reinvented/ > > I am posting this for two reason: > - Share AMPS approach to this problem and get the communities thoughts on > Spark's Structured Streaming approach if it will provide similar semantics > - Inquire about the availability of stream to stream joins and where I can > follow along (is there a JIRA or dev mailing list topic I can follow). > > Thank you in advance for any replies. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-Comparison-to-AMPS-tp27303.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
