Hi James,
Thanks for bringing this up.
The original design is to let users apply the prefix ONLY if they want to
override for the specific client. For example, for `metric.reporters` or
`interceptor.classes` which are defined for both producer and consumer (and
moving forward we are adding admin client as well), without the prefix they
will be applied to all embedded clients; and if users want to have
different config value overrides for different clients, they can use the
prefix.
So is you define for `metrics.reporter` it should be applied to both
producer and consumer (is it not the case for you? Please confirm).
For custom config k-v pairs, I think we did not discuss about this
in-depth. But our currently implementation is, indeed, restricting to
producer / consumer client prop definition only, for example.
getClientPropsWithPrefix(CONSUMER_PREFIX, ConsumerConfig.configNames());
props.putAll(originalsWithPrefix(prefix));
so if you do not add the prefix to your custom k-v pairs, it will be
skipped. Thinking about it a bit, I think we should probably do what you
were inclining to, i.e. to define custom k-v pairs also globally unless
overridden by prefix.
Could you file a JIRA for it?
Guozhang
On Wed, Nov 1, 2017 at 10:18 AM, James Cheng <wushuja...@gmail.com> wrote:
> Hi, we have a KafkaStreams app. We specify a custom metric reporter by
> doing:
>
> Properties config = new Properties();
> config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG,
> "kafka-broker1:9092");
> config.put(StreamsConfig.METRIC_REPORTER_CLASSES_CONFIG,
> "com.mycompany.MetricReporter");
> config.put("custom-key-for-metric-reporter", "value");
>
> Previously, our metric reporter would get passed the properties object
> upon instantiation, and would retrieve its custom config. It appears that
> in recent releases, that in order to apply my metric reporter to the
> consumer, that I have to specify my config as
> "consumer.custom-key-for-metric-reporter".
> And for the producer, I have to specify it as
> "producer.custom-key-for-metric-reporter".
> If I don't prefix it with "consumer." or "producer." , it appears it gets
> stripped out from the properties object that is passsed to my metric
> reporter when the consumer/producer gets initialized, and so my metric
> reporter can't get its config.
>
> That means that if I have a metrics reporter and I want to collect
> producer and consumer metrics, as well as kafka-streams metrics, that I
> have to specify my custom config 3 times:
> 1) consumer.custom-key-for-metric-reporter
> 2) producer.custom-key-for-metric-reporter
> 3) custom-key-for-metric-reporter
>
> Is that behavior as designed or is that a bug? What is the desired
> behavior for non-recognized keys in the properties object?
>
> And actually, for the metrics.reporter class itself, am I supposed to
> specify it as:
>
> metrics.reporter
>
> or
>
> metric.reporters
> producer.metric.reporters
> consumer.metric.reporters
>
> Thanks,
> -James
>
>
--
-- Guozhang
