berkaysynnada commented on PR #11330:
URL: https://github.com/apache/datafusion/pull/11330#issuecomment-2214018210
> > Thanks @xinlifoobar working on this issue.
> > What I suggest is that we can let people to define their own
normalizations. Rather than directing all values to lowercase, people can
declare their own normalization methods for specific option keys. If we choose
to do that in this PR, I believe the normalization process should be done
during the assignment of table options.
>
> The APIs that could be used in the `datafusion-sql` crate are limited. Do
you think it is a good idea to add a method, e.g.,
`get_options_normalized_function` a good idea?
`SqlParserOptions` is accessible, what do you think about adding it there?
But I'm not sure if we can support dynamic method registration. My idea is to
provide hard-coded functions like the one below, but I'm uncertain where to
bind it:
```
lazy_static! {
static ref NORMALIZATION_MAP: Mutex<HashMap<String, fn(String) ->
String>> = {
let mut m = HashMap::new();
m.insert("hf.user_access_token".to_string(), lowercase as fn(String)
-> String);
// Other normalizations
Mutex::new(m)
};
}
fn lowercase(s: String) -> String {
s.to_lowercase()
}
// Other normalizations
```
If the key is not found, use the default normalization or do nothing
(according to your `enable_options_value_normalization`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
