Hey Iceberg Community,

*TLDR*
I'd like to de-deprecate the 'write.metadata.path' view property
<https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/ViewProperties.java>
and would like to see if there are any objections.

*Details*
We have use-cases where it's necessary to configure tables and views to
have custom paths for metadata, in particular not to enforce adding the
metadata/ folder under the hood. For both tables and views there is
'write.metadata.path' property that could achieve this. However, for views
this is deprecated now.

After talking to Yufei, the reason for this property being deprecated is
that for views this is somewhat duplicated functionality with the
'location' property seemingly serving the same purpose. There are some
differences though:
  - 'location': This gives the root folder of the view and metadata would
be written under 'location'/metadata folder.
  - 'write.metadata.path': This has higher precedence over 'location' and
metadata would be written directly into 'write.metadata.path' without the
metadata/ folder.

An *alternative approach* could be to drop 'write.metadata.path' and change
the behaviour of 'location' to not create a metadata/ folder under the
hood. The view spec doesn't mention explicitly that metadata/ folder is a
requirement, however, it gives an example
<https://iceberg.apache.org/view-spec/#appendix-a-an-example> where it is
present and also mentions that it is there to follow the structure of the
tables. So changing this would be confusing at least, but could be
considered a breaking change too. I wouldn't go this way.

So long story short, I'd like to keep 'write.metadata.path' for views
despite the similar purpose as 'location', to avoid generating the
metadata/ folder.

Any feedback is appreciated!
Gabor

Reply via email to