Ok, thanks Fabian.
From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Tuesday, October 11, 2016 1:12 AM To: user@flink.apache.org Subject: Re: Wordindex conversation. Hi, you can do it like this: 1) you have to split each label record of the main dataset into separate records: (0,List(a, b, c, d, e, f, g)) -> (0, a), (0, b), (0, c), ..., (0, g) (1,List(b, c, f, a, g)) -> (1, b), (1, c), ..., (1, g) 2) join word index dataset with splitted main dataset: DataSet<Tuple2<Integer, String>> splittedMain = ... DataSet<Tuple2<Long, String>> wordIdx = ... DataSet<Integer, Long> joined = splittedMain.join(wordIdx).where(1).equalTo(1).with(...) 3) Group by Label: DataSet<Tuple2<Integer, Long[]>> labelsWithIdx = joined.groupBy(0).reduceGroup(...) // collect all indexes in list / array Best, Fabian 2016-10-10 23:49 GMT+02:00 Kürşat Kurt <kur...@kursatkurt.com <mailto:kur...@kursatkurt.com> >: Hi; I have MainDataset (Label,WordList) : (0,List(a, b, c, d, e, f, g)) (1,List(b, c, f, a, g)) ..and, wordIndex dataset(created with .zipWithIndex) : wordIndex> (0,a) wordIndex> (1,b) wordIndex> (2,c) wordIndex> (3,d) wordIndex> (4,e) wordIndex> (5,f) wordIndex> (6,g) How can i convert mainDataset to indexed wordList dataset like this: (0,List(1,2,3,4,5,6)) (1,List(2,3,5,0,6)
