Nicholas, are you interested in taking a stab at this? You could refer
https://github.com/apache/spark/commit/60472dbfd97acfd6c4420a13f9b32bc9d84219f3
2020년 1월 17일 (금) 오전 8:48, Takeshi Yamamuro <linguin....@gmail.com>님이 작성:
> The idea looks nice. I think web documents always help end users.
>
> Bests,
> Takeshi
>
> On Fri, Jan 17, 2020 at 4:04 AM Shixiong(Ryan) Zhu <
> shixi...@databricks.com> wrote:
>
>> "spark.sql("set -v")" returns a Dataset that has all non-internal SQL
>> configurations. Should be pretty easy to automatically generate a SQL
>> configuration page.
>>
>> Best Regards,
>> Ryan
>>
>>
>> On Wed, Jan 15, 2020 at 5:47 AM Hyukjin Kwon <gurwls...@gmail.com> wrote:
>>
>>> I think automatically creating a configuration page isn't a bad idea
>>> because I think we deprecate and remove configurations which are not
>>> created via .internal() in SQLConf anyway.
>>>
>>> I already tried this automatic generation from the codes at SQL built-in
>>> functions and I'm pretty sure we can do the similar thing for
>>> configurations as well.
>>>
>>> We could perhaps mimic what hadoop does
>>> https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/core-default.xml
>>>
>>> On Wed, 15 Jan 2020, 10:46 Sean Owen, <sro...@gmail.com> wrote:
>>>
>>>> Some of it is intentionally undocumented, as far as I know, as an
>>>> experimental option that may change, or legacy, or safety valve flag.
>>>> Certainly anything that's marked an internal conf. (That does raise
>>>> the question of who it's for, if you have to read source to find it.)
>>>>
>>>> I don't know if we need to overhaul the conf system, but there may
>>>> indeed be some confs that could legitimately be documented. I don't
>>>> know which.
>>>>
>>>> On Tue, Jan 14, 2020 at 7:32 PM Nicholas Chammas
>>>> <nicholas.cham...@gmail.com> wrote:
>>>> >
>>>> > I filed SPARK-30510 thinking that we had forgotten to document an
>>>> option, but it turns out that there's a whole bunch of stuff under
>>>> SQLConf.scala that has no public documentation under
>>>> http://spark.apache.org/docs.
>>>> >
>>>> > Would it be appropriate to somehow automatically generate a
>>>> documentation page from SQLConf.scala, as Hyukjin suggested on that ticket?
>>>> >
>>>> > Another thought that comes to mind is moving the config definitions
>>>> out of Scala and into a data format like YAML or JSON, and then sourcing
>>>> that both for SQLConf as well as for whatever documentation page we want to
>>>> generate. What do you think of that idea?
>>>> >
>>>> > Nick
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>>
>>>>
>
> --
> ---
> Takeshi Yamamuro
>
