Hi Timo, +1 for the proposal. A couple of remarks: - Are we going to enforce that the name is a valid class name? What is happening if it's not a correct name? - What are the implications of using a class that is not in the classpath in Table API? It looks to me that the name is metadata-only until we try to access the objects directly in Table/DataStream API. - Should Expressions.objectOf(String, Object... kv); also have an overload where you can put in the StructuredType in case where the class is not in the CP? - What is the expected outcome of supplying fewer keys than defined in the structured type? Are we going to make use of nullability here? If so, *_INSERT and *_REMOVE may have some use. - Talking about nullability: Is there some option to make the declared fields NOT NULL? If so, could you amend one example to show that? (Grammar implies that it's not possible) - One bigger concern is around the naming. For me, OBJECT is used for semi-structured types that are open. Your FLIP implies a closed design and that you want to add an open OBJECT later. I asked ChatGPT about other DB implementations and it seems like STRUCT is used more often (see below). -- So, I'd propose to call it STRUCT<...>, STRUCT_OF, structOf, UPDATE_STRUCT, and updateStruct respectively.
Best, Arvid SQL Engine Support for Structured/Object Types - PostgreSQL -- Type Support: Composite -- Definition: Closed -- Field Nullability: Yes - Oracle -- Type Support: Object -- Definition: Closed -- Field Nullability: Yes (default null) - SQL Server -- Type Support: UDT (CLR/Table) -- Definition: Closed -- Field Nullability: Yes - BigQuery -- Type Support: STRUCT -- Definition: Closed -- Field Nullability: Yes - Snowflake -- Type Support: OBJECT -- Definition: Open -- Field Nullability: Yes (dynamic keys) - MySQL -- Type Support: JSON workaround -- Definition: Open -- Field Nullability: Not enforced - DuckDB -- Type Support: STRUCT -- Definition: Closed -- Field Nullability: Yes - Spark SQL -- Type Support: STRUCT -- Definition: Closed -- Field Nullability: Yes - ClickHouse -- Type Support: TUPLE -- Definition: Closed -- Field Nullability: Yes On Tue, Apr 22, 2025 at 2:43 PM Gustavo de Morais <gustavopg...@gmail.com> wrote: > Thanks for the small but important changes proposed. IMO, these are > essential for dealing with these directly in SQL. > > On 2025/04/10 05:54:58 Timo Walther wrote: > > Hi everyone, > > > > I would like to start a discussion about FLIP-520: Simplify > > StructuredType handling [1]. > > > > Flink SQL already supports structured types in the engine, serializers, > > UDFs, and connector interfaces. However, currently only Table API was > > able to make use of them. While UDFs can take objects as input and > > return types, it is actually quite inconvenient to use them in > > transformations. > > > > This FLIP fixes some immediate blockers in the use of structured types. > > > > Looking forward to feedback. > > > > Cheers, > > Timo > > > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-520%3A+Simplify+StructuredType+handling > > > >
