If the Spark community doesn’t accept this solution, how about adding it as an extension in Iceberg? I’m also wondering what people here think about it.
Thanks for reviving the effort. Manu Szehon Ho <szehon.apa...@gmail.com>于2023年6月22日 周四00:45写道： > Hi, > > Yea, its definitely an issue. > > Fwiw, I was looking at reviving the old effort in Spark to pass in configs > dynamically in Spark SQL statement, which is probably the cleanest > solution. (https://github.com/apache/spark/pull/34072 was the old > effort, and I made https://github.com/apache/spark/pull/41683 based on > the suggestion from Spark community). Will keep the list posted. > > Thanks > Szehon > > On Fri, Jun 16, 2023 at 1:02 PM Wing Yew Poon <wyp...@cloudera.com.invalid> > wrote: > >> Hi, >> I recently put up a PR, https://github.com/apache/iceberg/pull/7790, to >> allow the write mode (copy-on-write/merge-on-read) to be specified in >> SQLConf. The use case is explained in the PR. >> Cheng Pan has an open PR, https://github.com/apache/iceberg/pull/7733, >> to allow locality to be specified in SQLConf. >> In the recent past, https://github.com/apache/iceberg/pull/6838/ was a >> PR to allow the write distribution mode to be specified in SQLConf. This >> was merged. >> Cheng Pan asks if there is any guidance on when we should allow configs >> to be specified in SQLConf. >> Thanks, >> Wing Yew >> >> ps. The above open PRs could use reviews by committers. >> >>
