If you use keys it might be possible to write a "no more messages" message to each partition.
Dave. > On May 29, 2016, at 01:51, David Olsen <davidolsen4...@gmail.com> wrote: > > I understand that the reception of Kafka messages may be out of > order[1][2]. But I have a multiple producers/ consumers scenario where I > would need to know when the last record is processed at the producer side, > so once consumers process e.g. NoMoreMessage, the entire system can react > such as stopping the consumers, proceeding to the next step, and so on. But > due to the nature of distributed system, there is no guaranteed that > NoMoreMessage issued at the producer side would be the last message > processed by consumers. So I need pieces of advice if any strategies I can > exploit in this scenario with Kafka? > > I appreciate any suggestions. > > [1]. > https://www.quora.com/Is-it-possible-to-consume-kafka-message-offset-based-on-timestamp > [2]. > http://stackoverflow.com/questions/35525356/kafka-multiple-producers-writing-to-same-topic-and-order-of-message-is-importa?rq=1 This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
