[
https://issues.apache.org/jira/browse/SPARK-57999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094267#comment-18094267
]
Hyukjin Kwon commented on SPARK-57999:
--------------------------------------
PR: https://github.com/apache/spark/pull/57079
> Add a generated, test-backed table of accepted input types for built-in
> scalar functions
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-57999
> URL: https://issues.apache.org/jira/browse/SPARK-57999
> Project: Spark
> Issue Type: Test
> Components: SQL, Tests
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Priority: Minor
> Labels: pull-request-available
>
> Spark's built-in functions do not consistently document which input DataTypes
> each argument accepts. The authoritative constraint lives only on the
> catalyst expression (inputTypes / checkInputDataTypes), and the public
> functions APIs (functions.scala, pyspark.sql.functions) are name-based
> facades over it.
> As a foundation for documenting accepted types (and per the principle that we
> should not document what is not tested), this adds
> FunctionAcceptedTypesSuite: an exhaustive, generated suite that, for every
> built-in scalar function and every argument position, probes each candidate
> DataType through the analyzer and records whether it is accepted (declared
> natively via ExpectsInputTypes vs. only via implicit cast), rejected
> (DATATYPE_MISMATCH), or inconclusive (other analysis errors). Results are
> written to a golden file,
> sql/core/src/test/resources/sql-functions/sql-function-accepted-types.md,
> regenerated with SPARK_GENERATE_GOLDEN_FILES=1.
> The suite is tagged @ExtendedSQLTest so it is skipped in the default test run
> (it probes every function against every type) and runs only in the dedicated
> extended-tests CI leg.
> This is the first step; follow-ups will use the golden table to drive and
> verify accepted-type documentation in SQL @ExpressionDescription, Scala
> Scaladoc, and PySpark docstrings.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
