Re: Iceberg's type system

2020-09-11 Thread Chen Song
sting type with a higher custom type (in form of annotation), as long as there is a base type (something like byte array maybe?) we can rely on. Chen On Fri, Sep 11, 2020 at 1:35 PM Ryan Blue wrote: > Sorry I missed this email! > > Right now, we don't support extending Iceberg&#x

Re: Iceberg's type system

2020-09-11 Thread Ryan Blue
Sorry I missed this email! Right now, we don't support extending Iceberg's type system. We are currently targeting a small set of types so that engines can easily implement them. And we've already hit some issues with the types that Iceberg supports: Spark, for example, doesn&#

Re: Iceberg's type system

2020-09-11 Thread Chen Song
Any thoughts, or suggestions on this? On Tue, Sep 8, 2020 at 3:01 PM Chen Song wrote: > Hi > > I have a general question on Iceberg's data type system. Iceberg has a > well defined type spec > which can be > mapped to types in Avro, Parqu

Iceberg's type system

2020-09-08 Thread Chen Song
Hi I have a general question on Iceberg's data type system. Iceberg has a well defined type spec which can be mapped to types in Avro, Parquet, ORC. If users want to use Iceberg and extend the universe of data types (e.g., adding custom typ