> Are we OK with catalog-provided metadata tables as a separate category?
I'm okay with providing metadata through a system table like this, as long as we think that people will want to access this data that way. Question 3, "If no, what should the SQL surface for labels be instead?" makes me think that a SQL surface is _assumed_ to be needed. I don't think it is necessarily the case that we need to expose these for SQL users. I thought that we wanted labels to expose additional context to engines for things like cost attribution logs or attaching an engine's policy to a table. That doesn't require a table-like user surface. I'm fine adding a metadata table if there's a use for it, but if we don't need one then it's simpler not to add and maintain it. And that avoids needing to answer questions like this as well. On Fri, Sep 25, 2026 at 1:13 PM Andrei Tserakhau via dev < [email protected]> wrote: > Hi all, > > Labels in the REST spec recently landed [1]. A catalog can now expose > object-level and per-field labels on load-table responses. As one of the > follow-ups, there is a proposal to add a .labels metadata table backed by > catalog data. > > During discussion of the follow-ups ([3], [4]), Peter raised a good > question on [3]: every metadata table today is derived from table metadata. > .labels would be different because the data comes from the catalog, may > vary by catalog, and may be absent if the catalog has nothing to return. > > I think this is less about Labels itself and more about a new *kind* of > data we would expose: metadata owned by the catalog rather than by storage. > Labels would be the first example, but later the same pattern could be used > to expose other catalog information as metadata tables. > > So the broader question is: do we want metadata tables to also expose > catalog-provided information? > > I think labels are a reasonable first case. They are structured, useful to > query via SQL, and fit the same access pattern as .snapshots or > .partitions. The table can stay read-only, limited to the spec-defined > shape, and empty when the catalog returns nothing. > > The tradeoff is that this breaks the current assumption that metadata > tables are deterministic projections of table metadata. It’s not explicitly > written anywhere, but that assumption exists today. > > To summarize the questions: > > 1. > > Are we OK with catalog-provided metadata tables as a separate category? > 2. > > If yes, should we mark them somehow so they are clearly different from > spec-backed metadata tables? > 3. > > If no, what should the SQL surface for labels be instead? > > Thanks, > > Andrei > > [1] REST spec labels: https://github.com/apache/iceberg/pull/15750 > [2] SupportsLabels: https://github.com/apache/iceberg/pull/18046 > [3] Labels metadata table: https://github.com/apache/iceberg/pull/18048 > [4] Spark DESCRIBE: https://github.com/apache/iceberg/pull/18049 > [5] Design: > https://docs.google.com/document/d/1aj-6JlfBiMYEEVtNuh5WLMOrRQiMCcyYUGbouPM4hXI/edit?tab=t.0#heading=h.2w0kmp1v1gwv >
