Hi, I don't think that the `producer.properties` file exist once per broker, because this file should not be at the broker machine at all. It should be on the application side (ie, your "modules").
For transactions to work correctly, each `Producer` instance needs a cluster wide unique transactional.id. I hope this answers your questions. Btw: you might also be interested in this two blog posts: - https://www.confluent.io/blog/transactions-apache-kafka/ - https://www.confluent.io/blog/enabling-exactly-kafka-streams/ -Matthias On 5/14/18 7:26 AM, thomas.k...@t-systems.com wrote: > Hi, > > the following article describes how to configure Apache Kafka system to > enforce exactly once approach: > > https://www.confluent.io/blog/exactly-once-semantics-are-possible-heres-how-apache-kafka-does-it/ > > One recommendation states that parameter transactional.id should be set in > producer.properties file which usually exists only once per broker. > > Now I have the following questions. Our system consists of 4 modules which > interact in a sequential way by exchanging messages using Kafka topics > > > A --> B --> C --> D > > I.e. modules A, B and C act as message producers and modules B, C and D act > as message consumers (and all modules do some application specific useful > processing). > > 1) Do we need separate producer/consumer configurations for these > modules and if so where and how do we have to configure them (as producer > configuration files are related to the number of brokers and not to the > number of modules)? > 2) Does transcational.id "only" cover technical aspects of Apache Kafka > or does it cover application aspects as well (i.e. is my assumption which led > to question 1 correct)? > 3) In case it is sufficient to configure transactional.id parameter in > producer.properties files and we have more than one broker (and with that > more than one producer.properties file). Do the values of the parameter > transactional.id have to be equal or do there have to be different (pairwise > disjoint) values? > > I would be glad to get answers to these questions. > > Kind regards, > Thomas Kass > > > > > >
signature.asc
Description: OpenPGP digital signature
