Hi Qing, > am I right to think that there will be 1 reduce function per key, and they will never overlap?
I agree with this, please correct me if I'm wrong. For a certain key, it will be divided into a certain key group range, and thus sent to a certain sub-task(reduce function instance). A key is only processed by one sub-task of an operator, so the subtasks of one key don't overlap. But in turn, a subtask will process multiple keys. For example: if we have many pairs like: (k1,v11), ... (k2,v2) ... (k1,v12) ...(k3,v3) ... (k1,v13),... v12 is always processed after v11. Best, Yanfei Qing Lim <q....@mwam.com> 于2022年11月3日周四 16:17写道： > Hi Yanfei > > Thanks for the explanation. > > > > If I use reduce in the context of keyed stream with window, am I right to > think that there will be 1 reduce function per key, and they will never > overlap? Each reduce function instance will only receive elements from the > same key in order. > > > > *From:* Yanfei Lei <fredia...@gmail.com> > *Sent:* 03 November 2022 03:06 > *To:* Qing Lim <q....@mwam.com> > *Cc:* User <user@flink.apache.org> > *Subject:* Re: Does reduce function on keyed window gives any guarantee > on the order of elements? > > > > Hi Qing, > > > Does it guarantee that it will be called in the same order of elements > in the stream, where value2 is always 1 element after value1? > > > Order is maintained within each parallel stream partition. If the reduce > operator only has one sending- sub-task, the answer is YES, but if reduce > operator has multiple sending- sub-task, order among elements is only > preserved for each pair of sending- and receiving tasks. > > The answers under > https://stackoverflow.com/questions/38354713/ordering-of-records-in-stream > might > help. > > Best, > > Yanfei > > > > Qing Lim <q....@mwam.com> 于2022年11月3日周四 01:32写道： > > Hi, Flink User Group > > > > I am trying to use Reduce function, I wonder does it guarantee order when > its called? > > > > The signature is as follow: > > > > T reduce(T value1, T value2) throws Exception; > > > > Does it guarantee that it will be called in the same order of elements in > the stream, where value2 is always 1 element after value1? > > > > Kind regards > > > > > > *Qing Lim *| Marshall Wace LLP, George House, 131 Sloane Street, London | > E-mail: > q....@mwam.com | Tel: +44 207 925 4865 > > > > > > This e-mail and any attachments are confidential to the addressee(s) and > may contain information that is legally privileged and/or confidential. If > you are not the intended recipient of this e-mail you are hereby notified > that any dissemination, distribution, or copying of its content is strictly > prohibited. If you have received this message in error, please notify the > sender by return e-mail and destroy the message and all copies in your > possession. > > > To find out more details about how we may collect, use and share your > personal information, please see https://www.mwam.com/privacy-policy. > This includes details of how calls you make to us may be recorded in order > for us to comply with our legal and regulatory obligations. > > > To the extent that the contents of this email constitutes a financial > promotion, please note that it is issued only to and/or directed only at > persons who are professional clients or eligible counterparties as defined > in the FCA Rules. Any investment products or services described in this > email are available only to professional clients and eligible > counterparties. Persons who are not professional clients or eligible > counterparties should not rely or act on the contents of this email. > > > Marshall Wace LLP is authorised and regulated by the Financial Conduct > Authority. Marshall Wace LLP is a limited liability partnership registered > in England and Wales with registered number OC302228 and registered office > at George House, 131 Sloane Street, London, SW1X 9AT. If you are receiving > this e-mail as a client, or an investor in an investment vehicle, managed > or advised by Marshall Wace North America L.P., the sender of this e-mail > is communicating with you in the sender's capacity as an associated or > related person of Marshall Wace North America L.P. ("MWNA"), which is > registered with the US Securities and Exchange Commission ("SEC") as an > investment adviser. Registration with the SEC does not imply that MWNA or > its employees possess a certain level of skill or training. > > > > -- Best, Yanfei
