Hi, Yea, its definitely an issue.
Fwiw, I was looking at reviving the old effort in Spark to pass in configs dynamically in Spark SQL statement, which is probably the cleanest solution. (https://github.com/apache/spark/pull/34072 was the old effort, and I made https://github.com/apache/spark/pull/41683 based on the suggestion from Spark community). Will keep the list posted. Thanks Szehon On Fri, Jun 16, 2023 at 1:02 PM Wing Yew Poon <wyp...@cloudera.com.invalid> wrote: > Hi, > I recently put up a PR, https://github.com/apache/iceberg/pull/7790, to > allow the write mode (copy-on-write/merge-on-read) to be specified in > SQLConf. The use case is explained in the PR. > Cheng Pan has an open PR, https://github.com/apache/iceberg/pull/7733, to > allow locality to be specified in SQLConf. > In the recent past, https://github.com/apache/iceberg/pull/6838/ was a PR > to allow the write distribution mode to be specified in SQLConf. This was > merged. > Cheng Pan asks if there is any guidance on when we should allow configs to > be specified in SQLConf. > Thanks, > Wing Yew > > ps. The above open PRs could use reviews by committers. > >
