Thanks for starting this thread, and for all the great discussions.
I can chime in on use cases from the Microsoft OneLake perspective. We have
a requirement to customize the metadata path of the view metadata json
file. We want the file path to have a specific naming convention that can
be an arbitrary subdirectory name.
Right now, using the `location` parameter, `metadata/` is added
automatically as the subdirectory for the json file [1], ie
{location}/metadata/view1.metadata.json
We want to customize the name of the subdirectory with
`write.metadata.path` [2], so the view will be stored in
{location}/{write.metadata.path}/view1.metadata.json
In summary, I prefer option 2 to give writers more control over the exact
path of the metadata json. And this also follows the convention for
creating an Iceberg table with the write.metadata.path property.
Best,
Kevin Liu
[1]
https://github.com/apache/iceberg/blob/1bf44d9fed8b99dd73115ec31659266248cbb363/core/src/main/java/org/apache/iceberg/view/BaseViewOperations.java#L173
[2]
https://github.com/apache/iceberg/blob/1bf44d9fed8b99dd73115ec31659266248cbb363/core/src/main/java/org/apache/iceberg/view/BaseViewOperations.java#L166-L169
On Tue, Jan 27, 2026 at 2:43 PM Ryan Blue <[email protected]> wrote:
> I think it is okay to go with option 1, but I don't see a justification
> for why a change here is needed. Why break with existing convention when it
> should not matter to users what location is used for metadata?
>
> I'd prefer option 2, to let people configure this if they choose. But I
> think the best solution is to do nothing and encourage people not to care
> about where metadata files end up. Looks like the justification is "we have
> use cases". Could you share what those are?
>
> On Mon, Jan 26, 2026 at 1:27 PM Yufei Gu <[email protected]> wrote:
>
>> Given that the Iceberg community is moving toward a catalog based
>> approach, I think option 1 should work reasonably well. From a reader
>> perspective, there should be no behavior change, since the metadata path is
>> always resolved from the catalog, regardless of whether the underlying path
>> includes a metadata directory or not. For writers, as long as we converge
>> on the convention of not implicitly adding a metadata subdirectory, this
>> also seems acceptable. This is already how tables behave today. We have not
>> enforced a strict rule for table metadata paths in the spec, and different
>> impls have been able to interoperate without issues.
>>
>> Yufei
>>
>>
>> On Fri, Jan 23, 2026 at 11:35 AM Gábor Kaszab <[email protected]>
>> wrote:
>>
>>> Thank you All for your responses!
>>> Let me summarize what we have so far to see what potential next steps we
>>> can make.
>>>
>>> 1) Change 'location' property to not create a 'metadata/' subfolder
>>> - Doesn't seem to be a great issue that already written tables have
>>> 'metadata/' subfolder, while newer ones won't
>>> - Easy to implement in Iceberg code
>>> - Doing this will let us drop 'write.metadata.path' property
>>> My concern:
>>> - We only have control on Iceberg code, that means non-REST
>>> catalogs, but different REST catalog implementations could still create
>>> paths the old way.
>>> - Is there a way we can enforce this? Adding to spec seems overkill.
>>>
>>> 2) De-deprecate 'write.metadata.path'
>>> - Using this will let us to avoid having 'metadata/' folder in the
>>> path
>>> - Doc <https://iceberg.apache.org/docs/latest/view-configuration/>
>>> says "Base location for metadata files". Seems ambiguous if this means that
>>> we can have subfolders or not
>>> - Seems common sense to not add 'metadata/' folder, but there seems
>>> to be no strict guarantee that all REST catalog implementations follow this
>>>
>>> WDYT about next steps? Maybe go with 2) and be more explicit in the docs?
>>>
>>> Best Regards,
>>> Gabor Kaszab
>>>
>>> Ryan Blue <[email protected]> ezt írta (időpont: 2026. jan. 16., P,
>>> 1:03):
>>>
>>>> I think that the "non-catalog option" means that it works when you're
>>>> pointed at a particular metadata.json file. Commits always have to go
>>>> through some catalog-based mechanism (now that Hadoop/FS commits are
>>>> deprecated in the spec).
>>>>
>>>> On Thu, Jan 15, 2026 at 1:23 AM Péter Váry <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Yufei,
>>>>>
>>>>> I think keeping the non-catalog option available makes sense, and
>>>>> adding an extra check for this non-standard approach doesn’t seem like a
>>>>> big issue to me. That said, the community consensus appears to be
>>>>> gradually
>>>>> shifting away from the non-catalog approach. If you look at the REST
>>>>> planning mode discussion [1], we’re even considering Iceberg tables
>>>>> without
>>>>> any metadata stored on the underlying storage. It might be worth defining
>>>>> a
>>>>> consistent, general approach for decisions like these.
>>>>>
>>>>> Thanks,
>>>>> Peter
>>>>>
>>>>> [1] - [DISCUSS] REST: Scan Planning mode -
>>>>> https://lists.apache.org/thread/z1g4y8b4ogdrn0jjtjlgg7yjgxdbzpvg
>>>>>
>>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 15.,
>>>>> Cs, 0:17):
>>>>>
>>>>>> Peter,
>>>>>> Based on the discussion(sorry, I don't have a link to share) so far,
>>>>>> the community conclusion is that we should always keep the non catalog
>>>>>> option available. In other words, even catalogs are the recommended way
>>>>>> to
>>>>>> locate view metadata, non catalog based access remains a supported
>>>>>> option.
>>>>>> This gives us flexibility to support different deployment models.
>>>>>>
>>>>>> Yufei
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 14, 2026 at 2:20 AM Péter Váry <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> > For example, when locating the view metadata.json, clients would
>>>>>>> need to check two possible locations, "/location" and
>>>>>>> "/location/metadata".
>>>>>>>
>>>>>>> Do I understand correctly that users are expected to use the Catalog
>>>>>>> to locate metadata.json? In my opinion they should not rely on
>>>>>>> filenames to
>>>>>>> locate views.”
>>>>>>>
>>>>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 14.,
>>>>>>> Sze, 3:49):
>>>>>>>
>>>>>>>> Thanks Gabor for raising this. The alternative approach is
>>>>>>>> appealing, as it avoids supporting the same behavior through multiple
>>>>>>>> properties. My main concern is potential impact on existing use cases.
>>>>>>>> For
>>>>>>>> example, when locating the view metadata.json, clients would need to
>>>>>>>> check
>>>>>>>> two possible locations, "/location" and "/location/metadata". This may
>>>>>>>> not
>>>>>>>> be a significant issue, since the spec does not mandate the metadata
>>>>>>>> suffix, but it does introduce incompatibility with the existing
>>>>>>>> convention.
>>>>>>>> It would be helpful for others to chime in and share view use cases
>>>>>>>> that
>>>>>>>> rely on the "metadata" suffix today, and POV on whether it's OK to
>>>>>>>> adapt
>>>>>>>> the change.
>>>>>>>>
>>>>>>>> Yufei
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jan 13, 2026 at 5:23 AM Gábor Kaszab <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> Hey Iceberg Community,
>>>>>>>>>
>>>>>>>>> *TLDR*
>>>>>>>>> I'd like to de-deprecate the 'write.metadata.path' view property
>>>>>>>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/ViewProperties.java>
>>>>>>>>> and would like to see if there are any objections.
>>>>>>>>>
>>>>>>>>> *Details*
>>>>>>>>> We have use-cases where it's necessary to configure tables and
>>>>>>>>> views to have custom paths for metadata, in particular not to enforce
>>>>>>>>> adding the metadata/ folder under the hood. For both tables and views
>>>>>>>>> there
>>>>>>>>> is 'write.metadata.path' property that could achieve this. However,
>>>>>>>>> for
>>>>>>>>> views this is deprecated now.
>>>>>>>>>
>>>>>>>>> After talking to Yufei, the reason for this property being
>>>>>>>>> deprecated is that for views this is somewhat duplicated
>>>>>>>>> functionality with
>>>>>>>>> the 'location' property seemingly serving the same purpose. There are
>>>>>>>>> some
>>>>>>>>> differences though:
>>>>>>>>> - 'location': This gives the root folder of the view and
>>>>>>>>> metadata would be written under 'location'/metadata folder.
>>>>>>>>> - 'write.metadata.path': This has higher precedence over
>>>>>>>>> 'location' and metadata would be written directly into
>>>>>>>>> 'write.metadata.path' without the metadata/ folder.
>>>>>>>>>
>>>>>>>>> An *alternative approach* could be to drop 'write.metadata.path'
>>>>>>>>> and change the behaviour of 'location' to not create a metadata/
>>>>>>>>> folder
>>>>>>>>> under the hood. The view spec doesn't mention explicitly that
>>>>>>>>> metadata/
>>>>>>>>> folder is a requirement, however, it gives an example
>>>>>>>>> <https://iceberg.apache.org/view-spec/#appendix-a-an-example>
>>>>>>>>> where it is present and also mentions that it is there to follow the
>>>>>>>>> structure of the tables. So changing this would be confusing at
>>>>>>>>> least, but
>>>>>>>>> could be considered a breaking change too. I wouldn't go this way.
>>>>>>>>>
>>>>>>>>> So long story short, I'd like to keep 'write.metadata.path' for
>>>>>>>>> views despite the similar purpose as 'location', to avoid generating
>>>>>>>>> the
>>>>>>>>> metadata/ folder.
>>>>>>>>>
>>>>>>>>> Any feedback is appreciated!
>>>>>>>>> Gabor
>>>>>>>>>
>>>>>>>>
