Re: Apache Flink <=> Apache Ignite integration

2016-07-19 Thread Anton Vinogradov
Maitra > wrote: > > Hi > > I have raised a PR for the following scope. > > As a Flink source => run a continuous query against one or multiple > caches > > PR https://github.com/apache/ignite/pull/870 > Jira https://issues.apache.org/jira/browse/IGNITE-3303 > &g

Re: Apache Flink <=> Apache Ignite integration

2016-07-19 Thread Denis Magda
one or multiple > caches > > PR https://github.com/apache/ignite/pull/870 > Jira https://issues.apache.org/jira/browse/IGNITE-3303 > > Please review and share feedback. > > Regards > Saikat > > > On Mon, Apr 4, 2016 at 8:24 PM, Stephan Ewen wrote: > >

Re: Apache Flink <=> Apache Ignite integration

2016-07-16 Thread Saikat Maitra
Hi I have raised a PR for the following scope. As a Flink source => run a continuous query against one or multiple caches PR https://github.com/apache/ignite/pull/870 Jira https://issues.apache.org/jira/browse/IGNITE-3303 Please review and share feedback. Regards Saikat On Mon, Apr 4, 2

Re: Apache Flink <=> Apache Ignite integration

2016-04-05 Thread Stephan Ewen
Ignite Continuous > Queries as a source. If the user's data objects contain an indexed > ascending numeric or datetime field, we could use such a field as a > "position marker" by launching the query with the appropriate WHERE filter > when a replay is demanded. &g

Re: Apache Flink <=> Apache Ignite integration

2016-04-04 Thread Raul Kripalani
scending numeric or datetime field, we could use such a field as a "position marker" by launching the query with the appropriate WHERE filter when a replay is demanded. Do you have similar use cases with existing connectors? Cheers, *Raúl Kripalani* PMC & Committer @ Apache Ign

Re: Apache Flink <=> Apache Ignite integration

2016-04-04 Thread Raul Kripalani
ers" :) But once its there, I'm pretty sure people will use it. Cheers, *Raúl Kripalani* PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and Messaging Engineer http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani Blog: raul.io <http://rau

Re: Apache Flink <=> Apache Ignite integration

2016-04-04 Thread Stephan Ewen
Hi! - Sounds like the having Ignite for snapshots should work pretty much out of the box (via the IGFS) - The source and sink connector sounds like the next logical step. Does Ignite have a notion of stream partitions and offsets, to build a consistent replay around? This should probably have

Re: Apache Flink <=> Apache Ignite integration

2016-04-04 Thread Robert Metzger
Hi Raul, thanks a lot for reaching out to the Flink community. I'm really excited to see a Flink connector in Ignite. If you feel that the connector would be more suitable for our "connector library" feel free to open a JIRA and open a pull request. Were there requests in the Ignite community to

Re: Apache Flink <=> Apache Ignite integration

2016-03-31 Thread Saikat Maitra
Hi , I agree with Roman and Raul. https://issues.apache.org/jira/browse/IGNITE-813 allows injecting data to into cache via Data Streamer. Integrating with Ignite FileSystem for source and sink will allow for bidirectional connector. It will also allow easier implementation for DataStream transform

Re: Apache Flink <=> Apache Ignite integration

2016-03-31 Thread Aljoscha Krettek
Hi, it should already be possible to use the Ignite FileSystem to store state since we just use the HDFS FileSystem interface for that. Of course, one would have to properly set up the jars and paths and everything for Flink to pick up the IGFS classes. Cheers, Aljoscha On Wed, 30 Mar 2016 at 16:

Re: Apache Flink <=> Apache Ignite integration

2016-03-30 Thread Roman
Raul, Small comment from me. >* As a Flink sink => inject data directly into a cache via a DataStreamer. After reviews, IGNITE-813 is exactly this functionality. -Roman

Re: Apache Flink <=> Apache Ignite integration

2016-03-30 Thread Raul Kripalani
On Wed, Mar 30, 2016 at 2:20 PM, Roman wrote: > Raul, > > Small comment from me. > > >* As a Flink sink => inject data directly into a cache via a DataStreamer. > After reviews, IGNITE-813 is exactly this functionality. > > That's cool, Roman! The idea would be to host these (richer) modules as F

Apache Flink <=> Apache Ignite integration

2016-03-30 Thread Raul Kripalani
Hello from the Apache Ignite community! Last year there was an interesting thread [1] about such integration. Unfortunately there's been little follow-through, so let's try and fix that in 2016 ;-) I'm sure a lot has changed in the Flink community, with the recent graduation and

Re: Apache Ignite

2015-04-24 Thread Fabian Hueske
ed hash table is an interesting idea, but I don't see how it could be used as a computation backend for Flink. 2015-04-24 6:38 GMT+02:00 sirinath : > Any thoughts on this? > > > > -- > View this message in context: > http://apache-flink-mailing-list-archive.1008284.n3.na

Re: Apache Ignite

2015-04-23 Thread sirinath
Any thoughts on this? -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-Ignite-tp5115p5304.html Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Re: Apache Ignite

2015-04-17 Thread sirinath
I was thinking in the lines of using this as an alternative clustering / distributed computing engine. -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-Ignite-tp5115p5199.html Sent from the Apache Flink Mailing List archive. mailing list

Re: Apache Ignite

2015-04-16 Thread Fabian Hueske
interesting as well. 2015-04-12 11:53 GMT-05:00 sirinath : > I am wodering if tighter integration between Apache Ignite and Flink would > be > of benefit to both communities > > > > -- > View this message in context: > http://apache-flink-incubator-mailing-list-archive.1008284.

Apache Ignite

2015-04-12 Thread sirinath
I am wodering if tighter integration between Apache Ignite and Flink would be of benefit to both communities -- View this message in context: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Apache-Ignite-tp5115.html Sent from the Apache Flink (Incubator) Mailing List