Hi everyone and thanks for the proposal Moritz, The FLIP reads very well and I agree (mostly) with the design.
I have only one concern, which is about the representation of repeated elements as arrays. I think it is very important that a batch of XML documents with the same structure converts into an identical VARIANT structure. If I understand the current proposal correctly, this would not be the case for two documents: one with a single element and one with a repeated element. The first document would have a single node, while the second document would have an array. If you want to access the nodes, you would need a cumbersome CASE WHEN construct to handle both cases. It would be better to convert both documents into a VARIANT containing an ARRAY (one only holding a single element). Since deriving the right conversion strategy from a single XML document isn't possible, could we add an optional argument to pass conversion hints? There might be other solutions to this problem. IMO, it would also be fine to continue with this FLIP and address this issue later, but having a proposal for it would be good. Do you have any thoughts on this, Moritz? Best, Fabian On Mon, Sep 14, 2026 at 11:37 AM Timo Walther <[email protected]> wrote: > Hi Moritz, > > this is an excellent design document with the right level of detail. I > really like the design, it is a nice mixture of concerns. The result of > PARSE_XML looks and feels like a VARIANT while the XML content is > preserved in a lossless fashion and all attributes and text is > accessible. It is a nice combination of what other vendors offer (modulo > the historical legacy they have to deal with). > > Looking at various UDF implementations, XML handling is by far the most > important reason for the need of a custom function. So +1 for this FLIP. > > Thanks, > Timo > > On 11.09.26 14:54, Moritz Manner wrote: > > Hey everyone, > > > > I'd like to start a discussion on FLIP-612: Native XML Functions for > Flink > > SQL [1]. > > > > The goal is native XML parsing in Flink SQL, parsing XML into a VARIANT > > instead of requiring custom UDFs. > > > > The FLIP proposes three functions. PARSE_XML and TRY_PARSE_XML turn an > XML > > string into a VARIANT, and XML_STRING turns a VARIANT back into an XML > > string. > > This mirrors the existing JSON functions PARSE_JSON, TRY_PARSE_JSON, and > > JSON_STRING. > > > > Once XML is a VARIANT, you query it the same way as the output of > PARSE_JSON: > > with the variant accessors (variant.key, variant['key'], variant[index]) > > and a CAST to the type you need. > > > > The main part is the XML→VARIANT mapping: attributes and text go into @/$ > > fields, and since Variant objects store fields sorted by key rather than > in > > document order, a # field is used to recover the original order between > > differently-named siblings. Details and examples are in the FLIP. > > > > It's a small, additive API, no new SQL grammar or Table API methods > needed. > > > > Looking forward to your thoughts! > > > > Best, > > Moritz > > > > [1] > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/spaces/FLINK/pages/451974259/FLIP-612*Native*XML*Functions*for*Flink*SQL__;KysrKysr!!Ayb5sqE7!pusFK_SVIXLRWBYFy7nKxC-f5N7Uj1ePbB2T6nFANBnqs-0S_zubmv9UygMYHTQQ1LFe9LUt4tGSVL3srSc$ > >
