I think it is okay to go with option 1, but I don't see a justification for why a change here is needed. Why break with existing convention when it should not matter to users what location is used for metadata?
I'd prefer option 2, to let people configure this if they choose. But I think the best solution is to do nothing and encourage people not to care about where metadata files end up. Looks like the justification is "we have use cases". Could you share what those are? On Mon, Jan 26, 2026 at 1:27 PM Yufei Gu <[email protected]> wrote: > Given that the Iceberg community is moving toward a catalog based > approach, I think option 1 should work reasonably well. From a reader > perspective, there should be no behavior change, since the metadata path is > always resolved from the catalog, regardless of whether the underlying path > includes a metadata directory or not. For writers, as long as we converge > on the convention of not implicitly adding a metadata subdirectory, this > also seems acceptable. This is already how tables behave today. We have not > enforced a strict rule for table metadata paths in the spec, and different > impls have been able to interoperate without issues. > > Yufei > > > On Fri, Jan 23, 2026 at 11:35 AM Gábor Kaszab <[email protected]> > wrote: > >> Thank you All for your responses! >> Let me summarize what we have so far to see what potential next steps we >> can make. >> >> 1) Change 'location' property to not create a 'metadata/' subfolder >> - Doesn't seem to be a great issue that already written tables have >> 'metadata/' subfolder, while newer ones won't >> - Easy to implement in Iceberg code >> - Doing this will let us drop 'write.metadata.path' property >> My concern: >> - We only have control on Iceberg code, that means non-REST >> catalogs, but different REST catalog implementations could still create >> paths the old way. >> - Is there a way we can enforce this? Adding to spec seems overkill. >> >> 2) De-deprecate 'write.metadata.path' >> - Using this will let us to avoid having 'metadata/' folder in the >> path >> - Doc <https://iceberg.apache.org/docs/latest/view-configuration/> >> says "Base location for metadata files". Seems ambiguous if this means that >> we can have subfolders or not >> - Seems common sense to not add 'metadata/' folder, but there seems >> to be no strict guarantee that all REST catalog implementations follow this >> >> WDYT about next steps? Maybe go with 2) and be more explicit in the docs? >> >> Best Regards, >> Gabor Kaszab >> >> Ryan Blue <[email protected]> ezt írta (időpont: 2026. jan. 16., P, 1:03): >> >>> I think that the "non-catalog option" means that it works when you're >>> pointed at a particular metadata.json file. Commits always have to go >>> through some catalog-based mechanism (now that Hadoop/FS commits are >>> deprecated in the spec). >>> >>> On Thu, Jan 15, 2026 at 1:23 AM Péter Váry <[email protected]> >>> wrote: >>> >>>> Hi Yufei, >>>> >>>> I think keeping the non-catalog option available makes sense, and >>>> adding an extra check for this non-standard approach doesn’t seem like a >>>> big issue to me. That said, the community consensus appears to be gradually >>>> shifting away from the non-catalog approach. If you look at the REST >>>> planning mode discussion [1], we’re even considering Iceberg tables without >>>> any metadata stored on the underlying storage. It might be worth defining a >>>> consistent, general approach for decisions like these. >>>> >>>> Thanks, >>>> Peter >>>> >>>> [1] - [DISCUSS] REST: Scan Planning mode - >>>> https://lists.apache.org/thread/z1g4y8b4ogdrn0jjtjlgg7yjgxdbzpvg >>>> >>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 15., Cs, >>>> 0:17): >>>> >>>>> Peter, >>>>> Based on the discussion(sorry, I don't have a link to share) so far, >>>>> the community conclusion is that we should always keep the non catalog >>>>> option available. In other words, even catalogs are the recommended way to >>>>> locate view metadata, non catalog based access remains a supported option. >>>>> This gives us flexibility to support different deployment models. >>>>> >>>>> Yufei >>>>> >>>>> >>>>> On Wed, Jan 14, 2026 at 2:20 AM Péter Váry < >>>>> [email protected]> wrote: >>>>> >>>>>> > For example, when locating the view metadata.json, clients would >>>>>> need to check two possible locations, "/location" and >>>>>> "/location/metadata". >>>>>> >>>>>> Do I understand correctly that users are expected to use the Catalog >>>>>> to locate metadata.json? In my opinion they should not rely on filenames >>>>>> to >>>>>> locate views.” >>>>>> >>>>>> Yufei Gu <[email protected]> ezt írta (időpont: 2026. jan. 14., >>>>>> Sze, 3:49): >>>>>> >>>>>>> Thanks Gabor for raising this. The alternative approach is >>>>>>> appealing, as it avoids supporting the same behavior through multiple >>>>>>> properties. My main concern is potential impact on existing use cases. >>>>>>> For >>>>>>> example, when locating the view metadata.json, clients would need to >>>>>>> check >>>>>>> two possible locations, "/location" and "/location/metadata". This may >>>>>>> not >>>>>>> be a significant issue, since the spec does not mandate the metadata >>>>>>> suffix, but it does introduce incompatibility with the existing >>>>>>> convention. >>>>>>> It would be helpful for others to chime in and share view use cases that >>>>>>> rely on the "metadata" suffix today, and POV on whether it's OK to adapt >>>>>>> the change. >>>>>>> >>>>>>> Yufei >>>>>>> >>>>>>> >>>>>>> On Tue, Jan 13, 2026 at 5:23 AM Gábor Kaszab <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey Iceberg Community, >>>>>>>> >>>>>>>> *TLDR* >>>>>>>> I'd like to de-deprecate the 'write.metadata.path' view property >>>>>>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/ViewProperties.java> >>>>>>>> and would like to see if there are any objections. >>>>>>>> >>>>>>>> *Details* >>>>>>>> We have use-cases where it's necessary to configure tables and >>>>>>>> views to have custom paths for metadata, in particular not to enforce >>>>>>>> adding the metadata/ folder under the hood. For both tables and views >>>>>>>> there >>>>>>>> is 'write.metadata.path' property that could achieve this. However, for >>>>>>>> views this is deprecated now. >>>>>>>> >>>>>>>> After talking to Yufei, the reason for this property being >>>>>>>> deprecated is that for views this is somewhat duplicated functionality >>>>>>>> with >>>>>>>> the 'location' property seemingly serving the same purpose. There are >>>>>>>> some >>>>>>>> differences though: >>>>>>>> - 'location': This gives the root folder of the view and metadata >>>>>>>> would be written under 'location'/metadata folder. >>>>>>>> - 'write.metadata.path': This has higher precedence over >>>>>>>> 'location' and metadata would be written directly into >>>>>>>> 'write.metadata.path' without the metadata/ folder. >>>>>>>> >>>>>>>> An *alternative approach* could be to drop 'write.metadata.path' >>>>>>>> and change the behaviour of 'location' to not create a metadata/ folder >>>>>>>> under the hood. The view spec doesn't mention explicitly that metadata/ >>>>>>>> folder is a requirement, however, it gives an example >>>>>>>> <https://iceberg.apache.org/view-spec/#appendix-a-an-example> >>>>>>>> where it is present and also mentions that it is there to follow the >>>>>>>> structure of the tables. So changing this would be confusing at least, >>>>>>>> but >>>>>>>> could be considered a breaking change too. I wouldn't go this way. >>>>>>>> >>>>>>>> So long story short, I'd like to keep 'write.metadata.path' for >>>>>>>> views despite the similar purpose as 'location', to avoid generating >>>>>>>> the >>>>>>>> metadata/ folder. >>>>>>>> >>>>>>>> Any feedback is appreciated! >>>>>>>> Gabor >>>>>>>> >>>>>>>
