We are basically using kafka as a transport mechanism for multi-line log files.
So, for this we are using single partition topics (with a replica for good measure) writing to a multi-broker cluster. Our producer basically reads a file line-by-line (as it is being written to) and publishes each line as a message to the topic. We are also writing as quickly as we can (not waiting for ACK). What I am seeing is occasionally the messages in the topic appear to be slightly out of order when compared to the source file they were based on. I am wonder if this might happen when the producer switches brokers because we are not waiting for the ACK before continuing to write. Does this make any sense?? Thanks in advance! -Scott
