Hi,
what's the number of unique keys and the parallelism of your job? If the
former is larger than the latter you should indeed have one
"timeWindowFold" be responsible for several keys. How are you determining
whether one of these is only accumulating for a single key?
Cheers,
Aljoscha
On Mon, 5
On Mon, Sep 5, 2016 at 12:30 PM, Aljoscha Krettek
wrote:
> Hi,
> for this you would have to use a non-parallel window, i.e. something like
> stream.windowAll().apply(...). This does not compute per key but
> has the drawback that computation does not happen in parallel. If you only
> use it to co
Hi,
for this you would have to use a non-parallel window, i.e. something like
stream.windowAll().apply(...). This does not compute per key but
has the drawback that computation does not happen in parallel. If you only
use it to combine the pre-aggregated maps it could be OK, though.
Cheers,
Aljosc
On Fri, Sep 2, 2016 at 5:24 PM, Aljoscha Krettek
wrote:
> Hi,
> from this I would expect to get as many HashMaps as you have keys. The
> winFunction is also executed per-key so it cannot combine the HashMaps of
> all keys.
>
> Does this describe the behavior that you're seeing?
>
yes, it's the b
Hi,
from this I would expect to get as many HashMaps as you have keys. The
winFunction is also executed per-key so it cannot combine the HashMaps of
all keys.
Does this describe the behavior that you're seeing?
Cheers,
Aljoscha
On Fri, 2 Sep 2016 at 17:37 Luis Mariano Guerra
wrote:
> hi!
>
> I
hi!
I'm trying to collect some metrics by key per window and emiting the full
result at the end of the window to kafka, I started with a simple count by
key to test it but my requirements are a little more complex than that.
what I want to do is to fold the stream events as they come and then at
