Hi, Could you review my small PR [SPARK-49044][SQL] ValidateExternalType should return a child in error ( https://github.com/apache/spark/pull/47522 )? Changes contain tests that verify results.
TLDR: After fix error message will contain extra information: [B is not a valid external type for schema of string at getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, f3) If you need more information, please let me know. If you're busy, please let me know the best time to reach you again. On Mon, 29 Jul 2024 at 18:15, Mark Andreev <mark.andr...@gmail.com> wrote: > Hi Spark Devs, > > Please review my PR [ https://github.com/apache/spark/pull/47522 ] that > relates to ticket [ https://issues.apache.org/jira/browse/SPARK-49044 ]. > > Context: When we have mixed schema rows, the error message "{actual} is > not a valid external type for schema of {expected}" doesn't help to > understand the column with the problem. I suggest adding information about > the source column. > > Example: > https://github.com/mrk-andreev/example-spark-schema/blob/main/spark_4.0.0/src/test/scala/ErrorMsgSuite.scala > > Before fix: [B is not a valid external type for schema of string > After fix: [B is not a valid external type for schema of string at > getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, > true]), 1, f3) > > -- > Best regards, > Mark Andreev > -- Best regards, Mark Andreev
