Hi Team, I am facing an issue in finding the lag (unconsumed record count) from Kafka against a topic. I have another process that has to be run after all records are consumed.
Each topic has given a single partition with the replication factor as 1. Lag is computed by taking the difference of published offset and consumed offset. In a transactional scenario, the published offset which is the partition end offset includes control messages. In my understanding, this is internally added by Kafka to handle the transaction commit and rollback. I got count mismatches in a transaction rollback scenario. For a single unconsumed record, it is showing lag as 3. So is there any way I can exclude the control messages from the lag. Regards, Indu V This electronic mail (including any attachment thereto) may be confidential and privileged and is intended only for the individual or entity named above. Any unauthorized use, printing, copying, disclosure or dissemination of this communication may be subject to legal restriction or sanction. Accordingly, if you are not the intended recipient, please notify the sender by replying to this email immediately and delete this email (and any attachment thereto) from your computer system...Thank You.
