I guess explaining rationalization would be better to understanding the
situation.
It's related to skip converting params to lowercase before assigning to
Kafka parameter. (https://github.com/apache/spark/pull/23612) If we
guarantee lowercase key on interface(s) we can simply pass them to Kafka as
I wouldn't be opposed to also documenting that we canonicalize the keys as
lowercase, but the case-insensitivity is I think the primary property. It's
important to call out that data source developers don't have to worry about
a semantic difference between option("mykey", "value") and option("myKey
Hi All,
Given org.apache.spark.sql.sources.v2.DataSourceOptions which states the
following:
* An immutable string-to-string map in which keys are
case-insensitive. This is used to represent
* data source options.
Case-insensitivity can be reached many ways.The implementation provides
lowercase s
