oogle Cloud Dataflow provides distributed dataset which called PCollection, and syntactic sugar based on PCollection is provided in the form of "apply". Note that "apply" is different from spark api "map" which passing each element of the source through a function func. I wonder can spark support this kind of syntactic sugar, if not, why?
- spark core api vs. google cloud dataflow lonely Feb
- Re: spark core api vs. google cloud dataflow Reynold Xin