Hi Yun, I personally do not think the "beta" label is related to our commitment (or lack of it) in maintaining the API. IMHO, it means that the API is likely to undergo major changes.
I did not and do not really oppose removing the "beta" label from the Generic Tables API :) My suggestion in keeping it a bit longer was purely to allow more time for finding common use cases with the Table Sources proposal and possibly unifying some workflows. Having thought about that from that perspective some more, I actually agree that the Generic Tables API is _not_ likely to undergo major changes. If synergies with Table Sources can be found, most of the changes are probably going to happen in clients that currently use the Generic Tables API, the API itself is probably going to remain stable. That said, just for the sake of clarity, if we happen to need a major Generic Tables API change after removing the "beta" label, I believe we'd have to make a v2 of that API (which is ok from my POV per our evolution guidelines [1]). [1] https://polaris.apache.org/in-dev/unreleased/evolution/ Cheers, Dmitri. On Mon, Nov 17, 2025 at 8:20 PM yun zou <[email protected]> wrote: > Hi Dmitri, > > Thanks a lot for the feedback! > > I definitely agree that the current generic table support still has > limitations and that more work is needed. However, removing the “beta” > label doesn’t mean the feature is finalized. Instead, it signals to users > that we are committed to maintaining and improving it over time. Therefore, > I believe this will not block any future enhancements including extending > it to address the use cases described in the Table Source proposal. > > As Adam pointed out, we already have users trying it out and requesting > improvements. I believe this is a good time to remove the “beta” label to > encourage broader adoption and welcome new contributors who can help > advance the feature. > > Best Regards, > Yun > > On Mon, Nov 17, 2025 at 3:28 PM Adam Christian < > [email protected]> wrote: > > > I'm in favor of removing the "beta" label because it is starting to be > used > > by users. From the Slack feedback we have seen, there are users who > > have started to try out this feature. While they are still running into > > issues such as not having a Spark 4 plugin [1] and not having credential > > vending [2], there is real user usage indicating that folks find this > > beneficial. > > > > Now, I do agree with Dmitri that there are known limitations we need to > > handle that are impeding users. The two referenced examples above are > > obvious ones that need fixes to get wider adoption. I have been working > > with Yun & others to be able to solve some of these issues (like this PR > > [3]), but I agree that there may need to be more significant changes > > including a REST API change. > > > > From my understanding, during our last conversation in the community > about > > the "Table Sources" proposal [4], we decided we were going to analyze the > > Parquet file use case and see if Generic Tables can support this use case > > or whether we need to bring some ideas from the Table Sources proposal > into > > Generic Tables. I believe that this approach is still valid and will > > benefit our adopting users by identifying any enhancements with these new > > use cases. > > > > What do y'all think? > > > > [1] - https://github.com/apache/polaris/issues/3021 > > [2] - https://github.com/apache/polaris/issues/3020 > > [3] - https://github.com/apache/polaris/pull/3026 > > [4] - https://lists.apache.org/thread/652z1f1n2pgf3g2ow5y382wlrtnoqth0 > > > > Go community, > > > > Adam > > > > On Mon, Nov 17, 2025 at 12:23 PM Dmitri Bourlatchkov <[email protected]> > > wrote: > > > > > Hi Yun, > > > > > > We should indeed review the status of the Generic Tables API, so thanks > > for > > > starting this discussion! > > > > > > From my POV the key question is: how do we intend to proceed with known > > > limitations [1]? > > > > > > I believe the Table Sources proposal [2] addresses some (if not all) of > > > those limitations. It is certainly suitable for the same applications > > that > > > currently go through the Generic Tables API. > > > > > > I believe it would be wise to allow the Table Sources proposal to > develop > > > further to see if there are any synergies that can be leveraged with > > > respect to Generic Tables. If we removed the "beta" label from the > > Generic > > > Tables API now, it would make it harder for users to benefit from those > > > synergies later (due to virtually freezing the "spec"). > > > > > > At it stands now, the Generic Tables API is supported by Polaris, so > > > existing clients can continue operating normally. > > > > > > [1] > > > > > > > > > https://github.com/apache/polaris/blob/f056e22f7f3a7c53e233bef1b88d204d6a8e4d79/site/content/in-dev/unreleased/generic-table.md?plain=1#L162-L169 > > > > > > [2] https://lists.apache.org/thread/9f5b75fy25l9yzrtnlzqg6yh1bqdyjbt > > > > > > Thanks, > > > Dmitri. > > > > > > > > > On Fri, Nov 14, 2025 at 12:33 PM yun zou <[email protected]> > > > wrote: > > > > > > > Hi All, > > > > > > > > Generic Table has been available since Polaris 1.0 and has attracted > > > > interest from several users. We also have ongoing improvement and > > > extension > > > > work in progress, including Hudi support, Parquet support, and > > credential > > > > vending. > > > > > > > > Given this progress, I believe it’s a good time to remove the “beta” > > > label. > > > > If there are no objections, we will remove the “beta” label from the > > > > Generic Table in Polaris 1.3. > > > > > > > > Best Regards, > > > > > > > > Yun > > > > > > > > > >
