Re: Join operation on DStreams

2015-09-21 Thread guoxu1231
Thanks for the prompt reply. May I ask why the keyBy(f) is not supported in DStreams? any particular reason? or is it possible to add it in future release since that "stream.map(record => (keyFunction(record), record))" looks tedious. I checked the python source code, KeyBy looks like a "shortcu

Re: Join operation on DStreams

2015-09-20 Thread Reynold Xin
stream.map(record => (keyFunction(record), record)) For future reference, this question should go to the user list, not dev list. On Sun, Sep 20, 2015 at 11:47 PM, guoxu1231 wrote: > Hi Spark Experts, > > I'm trying to use join(otherStream, [numTasks]) on DStreams, and it > requires called o