Re: join with no element appearing in multiple join-pairs

2016-02-01 Thread Fridtjof Sander
I still have them. To be clear: I do need to compare each element with its successor, I just can't have one element to be paired multiple times in the same step. That's why I divide the join into two steps: data: x0 -> x1 -> x2 The first join will only pair (x0, x1). These may or may not be

Re: join with no element appearing in multiple join-pairs

2016-02-01 Thread Till Rohrmann
In the described case, can it be that you still have elements with `id % 2 == 1` in your data set or are they filtered out? If they are filtered out, then you can simply shift the indices for each iteration to the right. On Mon, Feb 1, 2016 at 12:32 PM, Fridtjof Sander < fsan...@mailbox.tu-berlin.

Re: join with no element appearing in multiple join-pairs

2016-02-01 Thread Fridtjof Sander
Hi Till, thanks for your reply! The problem with that is, that I sometimes combine two elements: So from x0 -> x1 -> x2 I join (x0, x1) which might become x0 -> x2 in the end. The indices from zipWithIndex then are 0 and 2, resulting in equal joins flags. Sequential elements always have to

Re: join with no element appearing in multiple join-pairs

2016-02-01 Thread Till Rohrmann
Hi Fridtjof, I might miss something, but can’t you assign the ids once before starting the iteration and then reuse them throughout the iterations? Of course you would have to add another field to your input data but then you don’t have to run the zipWithIndex for every iteration. Cheers, Till ​

Re: join with no element appearing in multiple join-pairs

2016-02-01 Thread Fridtjof Sander
(tried to reformat) Hi, I have a problem which seems to be unsolvable in Flink at the moment (1.0-Snapshot, current master branch) and I would kindly ask for some input, ideas on alternative approaches or just a confirmatory "yup, that doesn't work". ### Here's the situation: I have a datas

join with no element appearing in multiple join-pairs

2016-02-01 Thread Fridtjof Sander
Hi, I have a problem which seems to be unsolvable in Flink at the moment (1.0-Snapshot, current master branch) and I would kindly ask for some input, ideas on alternative approaches or just a confirmatory "yup, that doesn't work". ### Here's the situation: I have a dataset and its elements ar