I think union is what you are looking for.
Note that all data sets must be of the same type.
2016-05-18 16:15 GMT+02:00 Ritesh Kumar Singh <riteshoneinamill...@gmail.com
>:
> Hi,
>
> How can I perform a reduce operation on a group of datasets using Flink?
> Let's say my map function gives out n datasets: d1, d2, ... dN
> Now I wish to perform my reduce operation on all the N datasets at once
> and not on an individual level. The only way I figured out till now is
> using the union operator first like following:
>
> List<Dataset<X>> dataList = Arrays.asList(d1, d2, ... dN);
> Dataset<X> dFinal = null;
> for(Dataset<X> ds: dataList)
> {
> dFinal = dFinal.union(ds);
> }
> dFinal.groupBy(0).reduce(...);
>
> Is there a more efficient way of doing the above task using java APIs?
> GroupReduce only works on a single dataset at a time and I can't find any
> other methods that take multiple datasets as an input parameter.
>
> Thanks,
> --
> Ritesh Kumar Singh
> https://riteshtoday.wordpress.com/
>
