Hi I have raised a PR for the following scope.
As a Flink source => run a continuous query against one or multiple caches PR https://github.com/apache/ignite/pull/870 Jira https://issues.apache.org/jira/browse/IGNITE-3303 Please review and share feedback. Regards Saikat On Mon, Apr 4, 2016 at 8:24 PM, Stephan Ewen <se...@apache.org> wrote: > Hi! > > - Sounds like the having Ignite for snapshots should work pretty much out > of the box (via the IGFS) > - The source and sink connector sounds like the next logical step. Does > Ignite have a notion of stream partitions and offsets, to build a > consistent replay around? This should probably have its dedicated issue and > discussion thread. > > - For Ignite as an execution backend - I am not sure how relevant and > feasible that is. Many DataStream API features make use of the specific > Flink runtime. For streaming, the runtime is not as decoupled as for batch. > - I think the parameter server integration would not be part of the Flink > codebase - this is a pretty application specific thing that should be its > own project and it is actually not tightly coupled to Flink. > > Greetings, > Stephan > > > On Mon, Apr 4, 2016 at 4:35 PM, Robert Metzger <rmetz...@apache.org> > wrote: > > > Hi Raul, > > > > thanks a lot for reaching out to the Flink community. > > I'm really excited to see a Flink connector in Ignite. If you feel that > the > > connector would be more suitable for our "connector library" feel free to > > open a JIRA and open a pull request. > > > > Were there requests in the Ignite community to have an integration with > > Flink? > > > > > > > > On Thu, Mar 31, 2016 at 5:20 PM, Saikat Maitra <saikat.mai...@gmail.com> > > wrote: > > > > > Hi , > > > > > > I agree with Roman and Raul. > > > https://issues.apache.org/jira/browse/IGNITE-813 allows injecting data > > to > > > into cache via Data Streamer. Integrating with Ignite FileSystem for > > source > > > and sink will allow for bidirectional connector. It will also allow > > easier > > > implementation for DataStream transformations over Ignite FileSystem. > > > > > > Regards > > > Saikat > > > > > > On Thu, Mar 31, 2016 at 2:44 PM, Aljoscha Krettek <aljos...@apache.org > > > > > wrote: > > > > > > > Hi, > > > > it should already be possible to use the Ignite FileSystem to store > > state > > > > since we just use the HDFS FileSystem interface for that. Of course, > > one > > > > would have to properly set up the jars and paths and everything for > > Flink > > > > to pick up the IGFS classes. > > > > > > > > Cheers, > > > > Aljoscha > > > > > > > > On Wed, 30 Mar 2016 at 16:50 Raul Kripalani <ra...@apache.org> > wrote: > > > > > > > > > On Wed, Mar 30, 2016 at 2:20 PM, Roman <rsht...@yahoo.com.invalid> > > > > wrote: > > > > > > > > > > > Raul, > > > > > > > > > > > > Small comment from me. > > > > > > > > > > > > >* As a Flink sink => inject data directly into a cache via a > > > > > DataStreamer. > > > > > > After reviews, IGNITE-813 is exactly this functionality. > > > > > > > > > > > > > > > > > That's cool, Roman! The idea would be to host these (richer) > modules > > as > > > > > Flink connectors, like they do with others: > > > > > > > > > > > > https://github.com/apache/flink/tree/master/flink-streaming-connectors > > > > > https://github.com/apache/flink/tree/master/flink-batch-connectors > > > > > > > > > > > > > > >
