Hi Mark,
Added a comment to Jira to provide more clarity to Description
When encountering mixed schema rows, the current error message "{actual} is
not a valid external type for schema of {expected}" lacks sufficient detail
to identify the problematic column. This ambiguity hinders troubleshootin
Hi,
Could you review my small PR [SPARK-49044][SQL] ValidateExternalType should
return a child in error ( https://github.com/apache/spark/pull/47522 )?
Changes contain tests that verify results.
TLDR: After fix error message will contain extra information: [B is not a
valid external type for sch
+1 on this too
When I implemented "group by all", I introduced at least two subtle bugs
that many reviewers weren't able to catch and those two bugs would not have
been possible to introduce if we had a single pass analyzer. Single pass
can make the whole framework more robust.
On Tue, Aug 2
This sounds like a good idea!
The Analyzer is complex. The changes in the new Analyzer should not affect
the existing one. The users could add the QO rules and rely on the existing
structures and patterns of the logical plan trees generated by the current
one. The new Analyzer needs to generate t
I think this would be a great move to encourage all sorts of engines and
table formats to take advantage of variant type and make sure it remains
compatible between all those systems.
I strongly support this,
Russ
On Tue, Aug 20, 2024 at 8:06 AM Fokko Driesprong wrote:
> Hey everyone,
>
> I agr
Hey everyone,
I agree the Parquet project is a good place to host and evolve the spec (we
could store it in parquet-variant?). We would need to align this with the
Parquet project. Anyway, I'm familiar both with Iceberg and Parquet and
happy to help where needed.
Kind regards,
Fokko
Op ma 19 au
