This is my window:
reduceByKeyAndWindow(
new Function2<Integer, Integer, Integer>() {
@Override
public Integer call(Integer i1, Integer i2) { return i1 + i2; }
},
new Function2<Integer, Integer, Integer>() {
public Integer call(Integer i1, Integer i2) { return i1 - i2; }
},
new Duration(60 * 5 * 1000),
new Duration(1 * 1000)
);
> On Nov 6, 2014, at 18:37, Gwen Shapira <gshap...@cloudera.com> wrote:
>
> What's the window size? If the window is around 10 seconds and you are
> sending data at very stable rate, this is expected.
>
>
>
> On Thu, Nov 6, 2014 at 9:32 AM, Eduardo Costa Alfaia <e.costaalf...@unibs.it
>> wrote:
>
>> Hi Guys,
>>
>> I am doing some tests with Spark Streaming and Kafka, but I have seen
>> something strange, I have modified the JavaKafkaWordCount to use
>> ReducebyKeyandWindow and to print in the screen the accumulated numbers of
>> the words, in the beginning spark works very well in each interaction the
>> numbers of the words increase but after 12 a 13 sec the results repeats
>> continually.
>>
>> My program producer remain sending the words toward the kafka.
>>
>> Does anyone have any idea about this?
>>
>>
>> -------------------------------------------
>> Time: 1415272266000 ms
>> -------------------------------------------
>> (accompanied
>> them,6)
>> (merrier,5)
>> (it
>> possessed,5)
>> (the
>> treacherous,5)
>> (Quite,12)
>> (offer,273)
>> (rabble,58)
>> (exchanging,16)
>> (Genoa,18)
>> (merchant,41)
>> ...
>> -------------------------------------------
>> Time: 1415272267000 ms
>> -------------------------------------------
>> (accompanied
>> them,12)
>> (merrier,12)
>> (it
>> possessed,12)
>> (the
>> treacherous,11)
>> (Quite,24)
>> (offer,602)
>> (rabble,132)
>> (exchanging,35)
>> (Genoa,36)
>> (merchant,84)
>> ...
>> -------------------------------------------
>> Time: 1415272268000 ms
>> -------------------------------------------
>> (accompanied
>> them,17)
>> (merrier,18)
>> (it
>> possessed,17)
>> (the
>> treacherous,17)
>> (Quite,35)
>> (offer,889)
>> (rabble,192)
>> (the
>> bed,1)
>> (exchanging,51)
>> (Genoa,54)
>> ...
>> -------------------------------------------
>> Time: 1415272269000 ms
>> -------------------------------------------
>> (accompanied
>> them,17)
>> (merrier,18)
>> (it
>> possessed,17)
>> (the
>> treacherous,17)
>> (Quite,35)
>> (offer,889)
>> (rabble,192)
>> (the
>> bed,1)
>> (exchanging,51)
>> (Genoa,54)
>> ...
>>
>> -------------------------------------------
>> Time: 1415272270000 ms
>> -------------------------------------------
>> (accompanied
>> them,17)
>> (merrier,18)
>> (it
>> possessed,17)
>> (the
>> treacherous,17)
>> (Quite,35)
>> (offer,889)
>> (rabble,192)
>> (the
>> bed,1)
>> (exchanging,51)
>> (Genoa,54)
>> ...
>>
>>
>> --
>> Informativa sulla Privacy: http://www.unibs.it/node/8155
>>
--
Informativa sulla Privacy: http://www.unibs.it/node/8155
