[
https://issues.apache.org/jira/browse/KAFKA-19213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pas Filip updated KAFKA-19213:
------------------------------
Description:
Using Kafka Java client, when creating a kafka consumer/producer using a
properties object with defaults the defaults are ignored.
Not sure if this is an intentional design choice or not but it was a surprise
to me.
My expectation was if a KafkaConsumer or KafkaProducer accepts a Properties
object it should not ignore the defaults in the Properties object.
The culprit seems to be in the propsToMap in the Utils class.
It doesn't use the propertyNames method to get all the keys known to the
Properties object.
{code:java}
public static Map<String, Object> propsToMap(Properties properties) {
Map<String, Object> map = new HashMap<>(properties.size());
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
if (entry.getKey() instanceof String) {
String k = (String) entry.getKey();
map.put(k, properties.get(k));
} else {
throw new ConfigException(entry.getKey().toString(),
entry.getValue(), "Key must be a string.");
}
}
return map;
}
{code}
I was using version 7.6.0-ccs of the kafka client, on the trunk branch the
implementation seems different but is subject to the same issue:
{code:java}
/**
* Convert a properties to map. All keys in properties must be string type.
Otherwise, a ConfigException is thrown.
* @param properties to be converted
* @return a map including all elements in properties
*/
public static Map<String, Object> propsToMap(Properties properties) {
return castToStringObjectMap(properties);
}
/**
* Cast a map with arbitrary type keys to be keyed on String.
* @param inputMap A map with unknown type keys
* @return A map with the same contents as the input map, but with String
keys
* @throws ConfigException if any key is not a String
*/
public static Map<String, Object> castToStringObjectMap(Map<?, ?> inputMap)
{
Map<String, Object> map = new HashMap<>(inputMap.size());
for (Map.Entry<?, ?> entry : inputMap.entrySet()) {
if (entry.getKey() instanceof String) {
String k = (String) entry.getKey();
map.put(k, entry.getValue());
} else {
throw new ConfigException(String.valueOf(entry.getKey()),
entry.getValue(), "Key must be a string.");
}
}
return map;
}{code}
A possible suggestion for how to fix the behavior:
{code:java}
public static Map<String, Object> propsToMap(Properties properties) {
final Enumeration<?> enumeration = properties.propertyNames();
Map<String, Object> props = new HashMap<>();
while (enumeration.hasMoreElements()) {
Object key = enumeration.nextElement();
if (key instanceof String) {
final String keyString = (String) key;
props.put(keyString,properties.getProperty(keyString));
} else {
throw new ConfigException(String.valueOf(key),properties.get(key),
"Key must be a string.");
}
}
return props;
}
{code}
Provided a basic test case to demonstrate the issue. See file attachment.
was:
Using Kafka Java client, when creating a kafka consumer/producer using a
properties object with defaults the defaults are ignored.
Not sure if this is an intentional design choice or not but it was a surprise
to me.
My expectation was if a KafkaConsumer or KafkaProducer accepts a Properties
object it should not ignore the defaults in the Properties object.
The culprit seems to be in the propsToMap in the Utils class.
It doesn't use the propertyNames method to get all the keys known to the
Properties object.
{code:java}
public static Map<String, Object> propsToMap(Properties properties) {
Map<String, Object> map = new HashMap<>(properties.size());
for (Map.Entry<Object, Object> entry : properties.entrySet()) {
if (entry.getKey() instanceof String) {
String k = (String) entry.getKey();
map.put(k, properties.get(k));
} else {
throw new ConfigException(entry.getKey().toString(),
entry.getValue(), "Key must be a string.");
}
}
return map;
}
{code}
On trunk branch implementation seems different but is subject to the same issue:
{code:java}
/**
* Convert a properties to map. All keys in properties must be string type.
Otherwise, a ConfigException is thrown.
* @param properties to be converted
* @return a map including all elements in properties
*/
public static Map<String, Object> propsToMap(Properties properties) {
return castToStringObjectMap(properties);
}
/**
* Cast a map with arbitrary type keys to be keyed on String.
* @param inputMap A map with unknown type keys
* @return A map with the same contents as the input map, but with String
keys
* @throws ConfigException if any key is not a String
*/
public static Map<String, Object> castToStringObjectMap(Map<?, ?> inputMap)
{
Map<String, Object> map = new HashMap<>(inputMap.size());
for (Map.Entry<?, ?> entry : inputMap.entrySet()) {
if (entry.getKey() instanceof String) {
String k = (String) entry.getKey();
map.put(k, entry.getValue());
} else {
throw new ConfigException(String.valueOf(entry.getKey()),
entry.getValue(), "Key must be a string.");
}
}
return map;
}{code}
A possible suggestion for how to fix the behavior:
{code:java}
public static Map<String, Object> propsToMap(Properties properties) {
final Enumeration<?> enumeration = properties.propertyNames();
Map<String, Object> props = new HashMap<>();
while (enumeration.hasMoreElements()) {
Object key = enumeration.nextElement();
if (key instanceof String) {
final String keyString = (String) key;
props.put(keyString,properties.getProperty(keyString));
} else {
throw new ConfigException(String.valueOf(key),properties.get(key),
"Key must be a string.");
}
}
return props;
}
{code}
Provided a basic test case to demonstrate the issue. See file attachment.
> Kafka java client ignores default properties
> --------------------------------------------
>
> Key: KAFKA-19213
> URL: https://issues.apache.org/jira/browse/KAFKA-19213
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Reporter: Pas Filip
> Priority: Minor
> Attachments: TestMapCreationFromPropertiesUsingDefaults.java
>
>
> Using Kafka Java client, when creating a kafka consumer/producer using a
> properties object with defaults the defaults are ignored.
> Not sure if this is an intentional design choice or not but it was a surprise
> to me.
> My expectation was if a KafkaConsumer or KafkaProducer accepts a Properties
> object it should not ignore the defaults in the Properties object.
>
> The culprit seems to be in the propsToMap in the Utils class.
> It doesn't use the propertyNames method to get all the keys known to the
> Properties object.
> {code:java}
> public static Map<String, Object> propsToMap(Properties properties) {
> Map<String, Object> map = new HashMap<>(properties.size());
> for (Map.Entry<Object, Object> entry : properties.entrySet()) {
> if (entry.getKey() instanceof String) {
> String k = (String) entry.getKey();
> map.put(k, properties.get(k));
> } else {
> throw new ConfigException(entry.getKey().toString(),
> entry.getValue(), "Key must be a string.");
> }
> }
> return map;
> }
> {code}
> I was using version 7.6.0-ccs of the kafka client, on the trunk branch the
> implementation seems different but is subject to the same issue:
> {code:java}
> /**
> * Convert a properties to map. All keys in properties must be string
> type. Otherwise, a ConfigException is thrown.
> * @param properties to be converted
> * @return a map including all elements in properties
> */
> public static Map<String, Object> propsToMap(Properties properties) {
> return castToStringObjectMap(properties);
> }
> /**
> * Cast a map with arbitrary type keys to be keyed on String.
> * @param inputMap A map with unknown type keys
> * @return A map with the same contents as the input map, but with String
> keys
> * @throws ConfigException if any key is not a String
> */
> public static Map<String, Object> castToStringObjectMap(Map<?, ?>
> inputMap) {
> Map<String, Object> map = new HashMap<>(inputMap.size());
> for (Map.Entry<?, ?> entry : inputMap.entrySet()) {
> if (entry.getKey() instanceof String) {
> String k = (String) entry.getKey();
> map.put(k, entry.getValue());
> } else {
> throw new ConfigException(String.valueOf(entry.getKey()),
> entry.getValue(), "Key must be a string.");
> }
> }
> return map;
> }{code}
>
> A possible suggestion for how to fix the behavior:
>
> {code:java}
> public static Map<String, Object> propsToMap(Properties properties) {
> final Enumeration<?> enumeration = properties.propertyNames();
> Map<String, Object> props = new HashMap<>();
> while (enumeration.hasMoreElements()) {
> Object key = enumeration.nextElement();
> if (key instanceof String) {
> final String keyString = (String) key;
> props.put(keyString,properties.getProperty(keyString));
> } else {
> throw new
> ConfigException(String.valueOf(key),properties.get(key), "Key must be a
> string.");
> }
> }
> return props;
> }
> {code}
> Provided a basic test case to demonstrate the issue. See file attachment.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
