It seems the original DISCUSS thread was split into two. I will answer
in the original thread for improving the history.
Cheers,
Timo
On 22.04.25 15:23, Arvid Heise wrote:
Hi Timo,
+1 for the proposal. A couple of remarks:
- Are we going to enforce that the name is a valid class name? What is
happening if it's not a correct name?
- What are the implications of using a class that is not in the classpath
in Table API? It looks to me that the name is metadata-only until we try to
access the objects directly in Table/DataStream API.
- Should Expressions.objectOf(String, Object... kv); also have an overload
where you can put in the StructuredType in case where the class is not in
the CP?
- What is the expected outcome of supplying fewer keys than defined in the
structured type? Are we going to make use of nullability here? If so,
*_INSERT and *_REMOVE may have some use.
- Talking about nullability: Is there some option to make the declared
fields NOT NULL? If so, could you amend one example to show that? (Grammar
implies that it's not possible)
- One bigger concern is around the naming. For me, OBJECT is used for
semi-structured types that are open. Your FLIP implies a closed design and
that you want to add an open OBJECT later. I asked ChatGPT about other DB
implementations and it seems like STRUCT is used more often (see below).
-- So, I'd propose to call it STRUCT<...>, STRUCT_OF, structOf,
UPDATE_STRUCT, and updateStruct respectively.
Best,
Arvid
SQL Engine Support for Structured/Object Types
- PostgreSQL
-- Type Support: Composite
-- Definition: Closed
-- Field Nullability: Yes
- Oracle
-- Type Support: Object
-- Definition: Closed
-- Field Nullability: Yes (default null)
- SQL Server
-- Type Support: UDT (CLR/Table)
-- Definition: Closed
-- Field Nullability: Yes
- BigQuery
-- Type Support: STRUCT
-- Definition: Closed
-- Field Nullability: Yes
- Snowflake
-- Type Support: OBJECT
-- Definition: Open
-- Field Nullability: Yes (dynamic keys)
- MySQL
-- Type Support: JSON workaround
-- Definition: Open
-- Field Nullability: Not enforced
- DuckDB
-- Type Support: STRUCT
-- Definition: Closed
-- Field Nullability: Yes
- Spark SQL
-- Type Support: STRUCT
-- Definition: Closed
-- Field Nullability: Yes
- ClickHouse
-- Type Support: TUPLE
-- Definition: Closed
-- Field Nullability: Yes
On Tue, Apr 22, 2025 at 2:43 PM Gustavo de Morais <gustavopg...@gmail.com>
wrote:
Thanks for the small but important changes proposed. IMO, these are
essential for dealing with these directly in SQL.
On 2025/04/10 05:54:58 Timo Walther wrote:
Hi everyone,
I would like to start a discussion about FLIP-520: Simplify
StructuredType handling [1].
Flink SQL already supports structured types in the engine, serializers,
UDFs, and connector interfaces. However, currently only Table API was
able to make use of them. While UDFs can take objects as input and
return types, it is actually quite inconvenient to use them in
transformations.
This FLIP fixes some immediate blockers in the use of structured types.
Looking forward to feedback.
Cheers,
Timo
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-520%3A+Simplify+StructuredType+handling
