Hey Ryanne, Is there any documentation where I can read more about this "slightly out-of-order records"? It would help very much to see how we can use MM2 in our systems.
Thanks, Peter On Thu, 23 Apr 2020 at 08:56, Péter Sinóros-Szabó < peter.sinoros-sz...@transferwise.com> wrote: > Hey Ryanna, > > Could the `max.in.flight.requests.per.connection=1` parameter help to > prevent the "slightly out-of-order records"? > Or is there any workaround for that? Duplicates are fine for me, but I'd > like to have the same order of messages too. > Can you please add some more detail about why those "slightly out-of-order > records" may happen? > > Thanks, > Peter > > On Wed, 22 Apr 2020 at 20:16, Ryanne Dolan <ryannedo...@gmail.com> wrote: > >> Hey Peter, Connect will need to support transactions before we can >> guarantee the order of records in remote topics. We can guarantee that no >> records are dropped or skipped, even during consumer failover/migration >> etc, but we can still have duplicates and slightly out-of-order records in >> the downstream remote topics, for now. >> >> Ryanne >> >> On Wed, Apr 22, 2020 at 3:39 AM Péter Sinóros-Szabó >> <peter.sinoros-sz...@transferwise.com.invalid> wrote: >> >> > Hey, >> > >> > so KIP-382 mentions that: >> > "Partitioning and order of records is preserved between source and >> remote >> > topics." >> > is the ordering of messages (I guess only in a partition) something >> that is >> > actually implemented in 2.4 (or in 2.5)? >> > >> > Or do I need to set `max.in.flight.requests.per.connection=1` ? >> > >> > Thanks, >> > Peter >> > >> >
