Flink Stream assign unique transaction ID

2016-02-19 Thread Tanguy Racinet
Hi, I am currently working on Flink streaming API and I am trying to assign unique IDs to transactions to track changes and mine Frequent Item Sets. I identified several potential solutions such as: - Check pointing Local variables (snapshotState and restoreState for once only passage) - Key

consume kafka stream with flink

2016-02-12 Thread Tanguy Racinet
Hello, I am currently trying to develop am algorithm mining frequent item sets over a data stream. I am using kafka to generate the stream, however I cannot manage to link Flink to Kafka. The code presented here is working but only using Flink version 0.9.1 https://github.com/dataArtisans/kafka

Re: flink loop

2015-02-09 Thread tanguy racinet
ted string. The far more serious problem is actually the cross > operation which is really costly. But if you need the cartesian product, > then you have to do it. > > On Sun, Feb 8, 2015 at 1:24 PM, tanguy racinet > wrote: > >> Hi, >> >> Thank you, that is

Re: flink loop

2015-02-08 Thread tanguy racinet
r. > > Cheers, > > Till > > On Sun, Feb 8, 2015 at 12:38 PM, tanguy racinet > wrote: > >> Hi, >> >> Thank you for you reply. It helped us solve the looping problems in a >> nicer way. >> >> We are struggling with some aspects of the cr

Re: flink loop

2015-02-08 Thread tanguy racinet
me examples in the documentation: > http://flink.apache.org/docs/0.8/programming_guide.html#iteration-operators > > Let us know if you have any questions! > > Cheers, > V. > > On 5 February 2015 at 20:37, tanguy racinet wrote: > >> Hi, >> >> We are trying t

flink loop

2015-02-05 Thread tanguy racinet
Hi, We are trying to develop the Apriori algorith with the Flink for our Data minning project. In our understanding, Flink could handle loop within the workflow. However, our knowledge is limited and we cannot find a nice way to do it. Here is the flow of my algorithm : GenerateCandidates > C