That’s my understanding as well, thanks for your confirmation.

From: Yanfei Lei <fredia...@gmail.com>
Sent: 04 November 2022 16:03
To: Qing Lim <q....@mwam.com>
Cc: User <user@flink.apache.org>
Subject: Re: Does reduce function on keyed window gives any guarantee on the 
order of elements?

Hi Qing,
> am I right to think that there will be 1 reduce function per key, and they 
> will never overlap?

I agree with this, please correct me if I'm wrong.  For a certain key, it will 
be divided into a certain key group range, and thus sent to a certain 
sub-task(reduce function instance). A key is only processed by one sub-task of 
an operator, so the subtasks of one key don't overlap.  But in turn, a subtask 
will process multiple keys.
For example: if we have many pairs like: (k1,v11), ... (k2,v2) ... (k1,v12) 
...(k3,v3) ... (k1,v13),...   v12 is always processed after v11.

Best,
Yanfei

Qing Lim <q....@mwam.com<mailto:q....@mwam.com>> 于2022年11月3日周四 16:17写道:
Hi Yanfei
Thanks for the explanation.

If I use reduce in the context of keyed stream with window, am I right to think 
that there will be 1 reduce function per key, and they will never overlap? Each 
reduce function instance will only receive elements from the same key in order.

From: Yanfei Lei <fredia...@gmail.com<mailto:fredia...@gmail.com>>
Sent: 03 November 2022 03:06
To: Qing Lim <q....@mwam.com<mailto:q....@mwam.com>>
Cc: User <user@flink.apache.org<mailto:user@flink.apache.org>>
Subject: Re: Does reduce function on keyed window gives any guarantee on the 
order of elements?

Hi Qing,
> Does it guarantee that it will be called in the same order of elements in the 
> stream, where value2 is always 1 element after value1?

Order is maintained within each parallel stream partition. If the reduce 
operator only has one sending- sub-task, the answer is YES, but if reduce 
operator has multiple sending- sub-task, order among elements is only preserved 
for each pair of sending- and receiving tasks.
The answers under 
https://stackoverflow.com/questions/38354713/ordering-of-records-in-stream 
might help.

Best,
Yanfei

Qing Lim <q....@mwam.com<mailto:q....@mwam.com>> 于2022年11月3日周四 01:32写道:
Hi, Flink User Group

I am trying to use Reduce function, I wonder does it guarantee order when its 
called?

The signature is as follow:

T reduce(T value1, T value2) throws Exception;

Does it guarantee that it will be called in the same order of elements in the 
stream, where value2 is always 1 element after value1?

Kind regards


Qing Lim | Marshall Wace LLP, George House, 131 Sloane Street, London | E-mail: 
q....@mwam.com<mailto:q....@mwam.com> | Tel: +44 207 925 4865




This e-mail and any attachments are confidential to the addressee(s) and may 
contain information that is legally privileged and/or confidential. If you are 
not the intended recipient of this e-mail you are hereby notified that any 
dissemination, distribution, or copying of its content is strictly prohibited. 
If you have received this message in error, please notify the sender by return 
e-mail and destroy the message and all copies in your possession.

To find out more details about how we may collect, use and share your personal 
information, please see https://www.mwam.com/privacy-policy. This includes 
details of how calls you make to us may be recorded in order for us to comply 
with our legal and regulatory obligations.

To the extent that the contents of this email constitutes a financial 
promotion, please note that it is issued only to and/or directed only at 
persons who are professional clients or eligible counterparties as defined in 
the FCA Rules. Any investment products or services described in this email are 
available only to professional clients and eligible counterparties. Persons who 
are not professional clients or eligible counterparties should not rely or act 
on the contents of this email.

Marshall Wace LLP is authorised and regulated by the Financial Conduct 
Authority. Marshall Wace LLP is a limited liability partnership registered in 
England and Wales with registered number OC302228 and registered office at 
George House, 131 Sloane Street, London, SW1X 9AT. If you are receiving this 
e-mail as a client, or an investor in an investment vehicle, managed or advised 
by Marshall Wace North America L.P., the sender of this e-mail is communicating 
with you in the sender's capacity as an associated or related person of 
Marshall Wace North America L.P. ("MWNA"), which is registered with the US 
Securities and Exchange Commission ("SEC") as an investment adviser.  
Registration with the SEC does not imply that MWNA or its employees possess a 
certain level of skill or training.




--
Best,
Yanfei

Reply via email to