Re: Parallelizing DataStream operations on Array elements

2016-11-07 Thread Till Rohrmann
In order to parallelize by voxel you have to do a keyBy(rowId) given that rowId is the same as voxel id. Glad to hear that you’ve resolved the problem :-) Cheers, Till ​ On Sat, Nov 5, 2016 at 2:47 AM, danielsuo wrote: > I was able to resolve my issue by collecting all the 'column' Arrays via

Re: Parallelizing DataStream operations on Array elements

2016-11-04 Thread danielsuo
I was able to resolve my issue by collecting all the 'column' Arrays via countWindowAll and using flatMap to emit 'row' Arrays. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Parallelizing-DataStream-operations-on-Array-elements-tp9911p9917.

Re: Parallelizing DataStream operations on Array elements

2016-11-04 Thread danielsuo
Till Rohrmann wrote > I'm not sure whether I grasp the whole problem, but can't you split > thevector up into the different rows, group by the row index and then > applysome kind of continuous aggregation or window function? So I could flatMap my incoming Arrays into (rowId, arrayElement) and gath

Re: Parallelizing DataStream operations on Array elements

2016-11-04 Thread Till Rohrmann
Hi Daniel, I'm not sure whether I grasp the whole problem, but can't you split the vector up into the different rows, group by the row index and then apply some kind of continuous aggregation or window function? Maybe it helps if you can share some of your code with the community to discuss the i