Small addition: The Flink Runner translates into the DataSet or DataStream API depending on the "streaming" flag of the PipelineOptions. The default mode is batch. Ultimately, this flag we be removed and replaced with an automated decision depending on the sources used.
On Thu, May 26, 2016 at 10:20 AM, Slim Baltagi <sbalt...@gmail.com> wrote: > Hi Ashutosh > > There is a related open JIRA: Enable DataSet and DataStream Joins > https://issues.apache.org/jira/browse/FLINK-2320 > > Slim > > > On May 26, 2016, at 3:05 AM, Fabian Hueske <fhue...@gmail.com> wrote: > > No, that is not supported yet. > Beam provides a common API but the Flink runner translates programs against > batch sources into the DataSet API programs and Beam programs against > streaming source into DataStream programs. > It is not possible to mix both. > > 2016-05-26 10:00 GMT+02:00 Ashutosh Kumar <kmr.ashutos...@gmail.com>: >> >> Thanks . So if we use Beam API with flink engine then we can get inter >> action between batch and stream ? As i know currently in flink Dataset and >> DStream can not talk . Is this correct ? >> Thanks >> Ashutosh >> >> >> On Thu, May 26, 2016 at 1:09 PM, Slim Baltagi <sbalt...@gmail.com> wrote: >>> >>> Hi Ashutosh >>> >>> Apache Beam provides a Unified API for batch and streaming. >>> It also supports multiple ‘runners’: local, Apache Spark, Apache Flink >>> and Google Cloud Data Flow (commercial service). >>> It is not an alternative to Flink because it is an API and you still need >>> an execution engine. >>> It can be used as an alternative API to using the two Flink APIs : >>> DataSet API and DataStream API. >>> It can be complementary to Flink in the way that you use Beam as API and >>> Flink as the execution engine. >>> Many of Flink committers are also Apache Beam committers! >>> The following blogs describe why Apache Beam: >>> >>> from Flink perspective: http://data-artisans.com/why-apache-beam/ >>> from Google perspective. >>> https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective >>> >>> >>> A few recent resources about Apache Beam published this month: May 2016 >>> >>> Running Apache Beam (screencast) >>> https://www.youtube.com/watch?v=dwxUbzbwtyI >>> Introduction to Apache Beam ( presentation) >>> https://skillsmatter.com/skillscasts/8036-apache-flink-may-meetup >>> Introduction to Apache Beam ( blog) >>> http://www.talend.com/blog/2016/05/02/introduction-to-apache-beam >>> >>> >>> I hope this helps. >>> >>> Thanks >>> >>> Slim Baltagi >>> >>> On May 26, 2016, at 2:20 AM, Ashutosh Kumar <kmr.ashutos...@gmail.com> >>> wrote: >>> >>> How does apache beam fits with flink ? Is it an alternative for flink or >>> complementary to it ? >>> >>> Thanks >>> Ashutosh >>> >>> >> > >
