Sorry I missed this email! Right now, we don't support extending Iceberg's type system. We are currently targeting a small set of types so that engines can easily implement them. And we've already hit some issues with the types that Iceberg supports: Spark, for example, doesn't support a time type, and very few engines support the UUID type.
That said, I think we may be able to support some form of logical types if all you need to do is to annotate that a certain value has a higher-level type. Like annotating that an Integer is actually an IP address. We've not thought about this very much, but it would be great to hear proposals. rb On Fri, Sep 11, 2020 at 10:30 AM Chen Song <chen.song...@gmail.com> wrote: > Any thoughts, or suggestions on this? > > On Tue, Sep 8, 2020 at 3:01 PM Chen Song <chen.song...@gmail.com> wrote: > >> Hi >> >> I have a general question on Iceberg's data type system. Iceberg has a >> well defined type spec >> <https://iceberg.apache.org/spec/#schemas-and-data-types> which can be >> mapped to types in Avro, Parquet, ORC. If users want to use Iceberg and >> extend the universe of data types (e.g., adding custom types on top of >> existing types), is it possible to do this w/o changing Iceberg core >> modules? >> >> Chen >> > > > -- > Chen Song > > -- Ryan Blue Software Engineer Netflix
