Hi, Maybe the messages of the same key should be in the *same partition* of Kafka topic
2018-07-29 11:01 GMT+08:00 Hequn Cheng <chenghe...@gmail.com>: > Hi harshvardhan, > If 1.the messages exist on the same topic and 2.there are no rebalance and > 3.keyby on the same field with same value, the answer is yes. > > Best, Hequn > > On Sun, Jul 29, 2018 at 3:56 AM, Harshvardhan Agrawal < > harshvardhan.ag...@gmail.com> wrote: > >> Hey, >> >> The messages will exist on the same topic. I intend to keyby on the same >> field. The question is that will the two messages be mapped to the same >> task manager and on the same slot. Also will they be processed in correct >> order given they have the same keys? >> >> On Fri, Jul 27, 2018 at 21:28 Hequn Cheng <chenghe...@gmail.com> wrote: >> >>> Hi Harshvardhan, >>> >>> There are a number of factors to consider. >>> 1. the consecutive Kafka messages must exist in a same topic of kafka. >>> 2. the data should not been rebalanced. For example, operators should be >>> chained in order to avoid rebalancing. >>> 3. if you perform keyBy(), you should keyBy on a field the consecutive >>> two messages share the same value. >>> >>> Best, Hequn >>> >>> On Sat, Jul 28, 2018 at 12:11 AM, Harshvardhan Agrawal < >>> harshvardhan.ag...@gmail.com> wrote: >>> >>>> Hi, >>>> >>>> >>>> We are currently using Flink to process financial data. We are getting >>>> position data from Kafka and we enrich the positions with account and >>>> product information. We are using Ingestion time while processing events. >>>> The question I have is: say I key the position datasream by account number. >>>> If I have two consecutive Kafka messages with the same account and product >>>> info where the second one is an updated position of the first one, does >>>> Flink guarantee that the messages will be processed on the same slot in the >>>> same worker? We want to ensure that we don’t process them out of order. >>>> >>>> Thank you! >>>> -- >>>> Regards, >>>> Harshvardhan >>>> >>> >>> -- >> Regards, >> Harshvardhan >> > > -- Blog:http://www.klion26.com GTalk:qcx978132955 一切随心
