[
https://issues.apache.org/jira/browse/SPARK-18433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-18433:
----------------------------------
Description:
Currently, `DataSource.scala` partially use `CaseInsensitiveMap` in code-path.
This issue aims to make `DataSource` to use `CaseInsensitiveMap` generally
except passing to the other modules (`InMemoryFileIndex` and
`InsertIntoHadoopFsRelationCommand`). They creates new case-sensitive
`HadoopConfs` by calling `newHadoopConfWithOptions(options)` inside.
Also, the following options should be consistent.
- CSVOptions
- FileStreamOptions
- JDBCOptions
- JSONOptions
- OrcOptions
- ParquetOptions
was:
Currently, `DataSource.scala` partially use `CaseInsensitiveMap` in code-path.
This issue aims to make `DataSource` to use `CaseInsensitiveMap` generally
except passing to the other modules (`InMemoryFileIndex` and
`InsertIntoHadoopFsRelationCommand`). They creates new case-sensitive
`HadoopConfs` by calling `newHadoopConfWithOptions(options)` inside.
Also, the following options should be consistent.
- CSVOptions
- JDBCOptions
- JSONOptions
- OrcOptions
- ParquetOptions
> Improve DataSource option keys to be more case-insensitive
> ----------------------------------------------------------
>
> Key: SPARK-18433
> URL: https://issues.apache.org/jira/browse/SPARK-18433
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Dongjoon Hyun
> Priority: Minor
>
> Currently, `DataSource.scala` partially use `CaseInsensitiveMap` in
> code-path.
> This issue aims to make `DataSource` to use `CaseInsensitiveMap` generally
> except passing to the other modules (`InMemoryFileIndex` and
> `InsertIntoHadoopFsRelationCommand`). They creates new case-sensitive
> `HadoopConfs` by calling `newHadoopConfWithOptions(options)` inside.
> Also, the following options should be consistent.
> - CSVOptions
> - FileStreamOptions
> - JDBCOptions
> - JSONOptions
> - OrcOptions
> - ParquetOptions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
