Hi all,
The documentation for the broadcast state explicitly says that the order of
broadcast events may differ across tasks, so the state updates should not
depend on a particular order. [1] But what to do in the use cases where the
order matters? Is there some way to enforce the order even at th
rethink on
> the source level already, e.g. change the method serialization to something
> which has a really lightweight parsing for finding the special events or
> such.
>
> Best regards
> Theo
>
> --
> *Von: *"Filip Niksic"
>
ng
> deserialization as much as possible to only recognize barrier events) and
> then add a downstream task with higher parallelism doing the full
> deserialization and other work?
> Best regardsTheo
> Ursprüngliche Nachricht
> Betreff: Re: [QUESTION] How to paralleliz
tely sure if my usage of state in the
> timestamp-and-watermark assigner and the mapper is correct. Is it possible
> for Flink to duplicate the assigner, move it around and somehow mess up the
> timestamps? Likewise, is it possible for things to go wrong with the mapper?
>
> Another
more elegantly, let me know!
Best regards,
Filip
On Tue, Oct 8, 2019 at 11:12 AM Filip Niksic wrote:
> Hi Chesnay,
>
> Thanks for the reply. While your solution ultimately does use multiple
> partitions, from what I can tell the underlying processing is still
> sequential. Imag
titionCustom(new BarrierPartitioner(), item -> item);
>
>
> On 08/10/2019 14:55, Filip Niksic wrote:
>
> Hi Yun,
>
> The behavior with increased parallelism should be the same as with no
> parallelism. In other words, for the input from the previous email, the
> output
ee 1 followed by 3 as output
>
> However, such an operator cannot be parallelized, since the order of
> Values and Barriers matters. That’s why I need to set parallelism to 1
> above. Is there a way to rewrite this to exploit parallelism?
>
> (Another reason to set parallelism to 1 above is that I’m assuming there
> is a single instance of the FlatMapFunction. A proper implementation would
> take more care in using state. Feel free to comment on that as well.)
>
>
> Best regards,
>
>
> Filip Niksic
>
>
>
proper implementation would
take more care in using state. Feel free to comment on that as well.)
Best regards,
Filip Niksic
Hi all,
I am part of a research group at the University of Pennsylvania. We are
researching distributed stream processing systems; in particular, we are
developing a framework for checking correctness of Flink programs through
differential testing.
What I want to ask you as a community of Flink u
