Re: [DISCUSS] Variant Spec Location

2024-08-16 Thread Gene Pang
>> If it's not an extension array then it will need to be a new first-class >> data type / layout and that will take some work & time to achieve support. >> However, I think that may be a prerequisite in some ways anyway. I believe >> (at least some) Iceberg

Re: [DISCUSS] Variant Spec Location

2024-08-16 Thread Gene Pang
Hi all, I am one of the main developers for Variant in Apache Spark. David Cashman (another one of the main Variant developers) and I have been working on Variant in Spark for a while, and we are excited by the interest from the Iceberg community! We have attended some of the Iceberg dev Variant