Good idea, JB! The new thread is here, https://lists.apache.org/thread/5nm0440bs0n9slvmhxhzg1mbnnvm5o7h
Yufei On Wed, Jul 8, 2026 at 7:39 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi folks, > > I believe the "façade" Semantic REST API should be the primary focus here, > as it represents the user-facing layer. The internal representation is less > of a concern since it can be updated or changed as needed. > > We should align on the API surface defined in the design document: > > https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.t6dop8lqx3uq > > My takeaways from this discussion are: > > 1. There seems to be no concern with introducing the new endpoints based > on /v1/{prefix}/namespaces/{ns}/semantic-models. > 2. The discussion is primarily centered on using "raw" JSON in the > endpoints. > > Personally, I think using raw JSON is acceptable because we can translate > it to our internal representation. However, we should consider that this > may introduce server-side overhead regarding marshalling/unmarshalling and > memory pressure. > > To reach a consensus, I suggest starting a fresh discussion thread > specifically focused on the REST API and the usage of JSON payloads. Yufei, > would you mind starting that thread? > > Thoughts? > > Regards, > JB > > > On Wed, Jul 8, 2026 at 3:53 AM Yufei Gu <[email protected]> wrote: > > > Thanks for the feedback. This is a good discussion. > > > > I'm fine with either representing the Ossie payload as a raw string or as > > an opaque JSON object in the REST spec. However, bringing the full Ossie > > JSON structure into the REST API is problematic because the Ossie schema > is > > versioned. Doing so would tightly couple the REST spec to Ossie > versions. I > > agree with Adam that keeping the payload opaque allows us to support > > different semantic model schemas without changing the REST API. We would > > only need to update the validator as the Ossie spec evolves. > > > > With this approach, whenever Ossie introduces a new version, Polaris > > doesn't need to change the REST spec. We would simply add support for the > > new version in the validator. > > > > I'm not sure I fully understand the client-side concern. Clients will > > follow the REST API regardless of whether the feature is in beta. In > fact, > > I think the beta status gives us more flexibility, since breaking changes > > are acceptable while the API is still evolving. > > > > Just in case it helps, the design doc [1] explains this: > > > > > Round trip is general byte-preserving, subject to JSON's key ordering > and > > > whitespace flexibility, and semantically lossless. Every field, > including > > > custom_extensions, survives write to read. > > > > [1] > > > > > https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.a2w9eycpqhof > > > > > > Yufei > > > > > > On Tue, Jul 7, 2026 at 7:57 AM Jean-Baptiste Onofré <[email protected]> > > wrote: > > > > > Hi > > > > > > I think we have to distinguish two aspects here: > > > 1. how do we "store" the semantic models in the Polaris service (json > > raw, > > > variant, object, ...). I'm not much concerned about that as it's our > > > internal plumbing. > > > 2. how the client will interact the semantic models, e.g. the façade. > > Here, > > > we must be cautious and reach a consensus. > > > > > > If I agree we can move forward on 1 without much concern (we can always > > > debate and change the internal representation), I suggest focusing on > > (2). > > > My preference would be to façade the Ossie JSON as an "object" (not as > a > > > raw string). It gives us the flexibility to change the "mapping" > between > > > the client request/response and our inner storage. > > > The API can manage the Ossie "objects" (create/update/resolve, e.g. > > > CRUD++). > > > > > > I suggest to focus on this part now, as it's really fundamental on the > > user > > > facing part. > > > > > > Regards > > > JB > > > > > > On Tue, Jul 7, 2026 at 4:41 PM Robert Stupp <[email protected]> wrote: > > > > > > > For beta, I think the minimum contract should be that the API makes > the > > > > intended client model explicit. > > > > > > > > If `semantic_model` stays a string, I think the docs/spec should say > > that > > > > clients should treat it as opaque round-trip document content, not > as a > > > > generated-client-modeled Ossie object. > > > > In that case, I also think we should define at least basic > operational > > > > bounds, for example, the maximum accepted document size and > validation > > > > expectations, because semantic documents can grow beyond small > > examples. > > > > > > > > If we expect clients to inspect or build UX around the payload > through > > > > generated clients, I would prefer modeling the Ossie document as JSON > > > > rather than escaped JSON in a string. > > > > > > > > Regarding discovery, I think beta should also state whether v1 is > > > > intentionally limited to namespace/name CRUD, or whether clients > should > > > > expect to discover usable models by version/type/capability later. > > > > I do not think we need to solve rich search now, but we should avoid > > > > implying more discoverability than the API provides. > > > > > > > > I agree this does not need to stop all semantic-model work, and I do > > not > > > > think it should block the beta wording cleanup in #4983. > > > > > > > > My main concern is that these points should be resolved before #4961 > > > merges > > > > as the first working CRUD API, because that is when client > expectations > > > > start to become sticky. > > > > > > > > On Tue, Jul 7, 2026 at 4:06 PM Adam Christian < > > > > [email protected]> wrote: > > > > > > > > > Thanks, Robert! > > > > > > > > > > Here are my thoughts: > > > > > 1. I'm fine with either making the REST API accept just the JSON > > string > > > > or > > > > > modeling it like an Ossie document. We just need to make a call. > > > > > - For the JSON string pros, we will be able to accept different > types > > > of > > > > > semantic model documents with different schemas without having to > > > update > > > > > the REST API. We would only have to update the validator. > > > > > - For the modeling pros, we will be more prescriptive about what we > > > > support > > > > > and what we do not inside the REST API. It would also decrease the > > > > > validator's workload. > > > > > For me, I could go either way. Do you have a preference? > > > > > > > > > > 2. I'll align the Semantic Model work with the registry endpoint. I > > can > > > > do > > > > > that now. I don't think that this improvement needs to slow the > > > Semantic > > > > > Model work. I'll complete that in parallel. > > > > > > > > > > 3. If you look into the Semantic Model design document, there is a > > > small > > > > > section about the OSI Version Evolution [1]. It doesn't lay out > > exactly > > > > > what you are asking for, but it states that we will support > multiple > > > > > versions of the OSI documents. You can save 0.1.1 or 0.2.0. Now, I > > > assume > > > > > we will support different versions of the specification and perform > > > light > > > > > validation on them. We probably will want to document a support > > matrix > > > > with > > > > > each version of Polaris as well. I don't think that this needs to > > stop > > > > > #4961, but I do think that we will need to speak about this before > we > > > > ship > > > > > it out to beta. Do you have any expectations that must be met > before > > we > > > > > ship? > > > > > > > > > > [1] - > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.n36d87je014l > > > > > > > > > > Go community, > > > > > > > > > > Adam > > > > > > > > > > On Tue, Jul 7, 2026 at 9:37 AM Robert Stupp <[email protected]> > wrote: > > > > > > > > > > > Thanks for the beta/Ossie cleanup in #4983. I think that is > useful > > > and > > > > > > should move forward. > > > > > > > > > > > > I do not think that fully closes the API-shape follow-up from > > #4816, > > > > > > though. Before #4961 turns the semantic-model API into real CRUD > > > > > behavior, > > > > > > I think we should still settle two things explicitly: > > > > > > > > > > > > 1. whether `semantic_model` should remain escaped JSON in a > string, > > > or > > > > > > whether the API should model the Ossie document as JSON; > > > > > > 2. whether endpoint advertisement should wait for / align with > the > > > > > endpoint > > > > > > registry work in #4926, so semantic-model discovery does not > > > hard-code > > > > > > another optional non-IRC API path. > > > > > > 3. how clients are expected to use the API in practice while it > is > > > > beta: > > > > > > whether clients should treat the document as opaque round-trip > > > storage, > > > > > > whether Polaris intends generated clients to model the Ossie > > payload, > > > > and > > > > > > what compatibility expectations clients should have before the > API > > > > > becomes > > > > > > stable. > > > > > > > > > > > > I support keeping semantic models as beta/preparatory Ossie > > document > > > > > > hosting. I just want to avoid making the beta API sticky before > the > > > > > payload > > > > > > and discovery shape are reviewed. > > > > > > > > > > > > On Mon, Jul 6, 2026 at 8:09 PM Yufei Gu <[email protected]> > > > wrote: > > > > > > > > > > > > > Hi folks, here is the PR to add the beta wording. Please take a > > > look, > > > > > > > https://github.com/apache/polaris/pull/4983. > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 11:53 PM Jean-Baptiste Onofré < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Yufei, Robert, > > > > > > > > > > > > > > > > I agree regarding the release process; shipping beta features > > is > > > > > > > consistent > > > > > > > > with our past practices. > > > > > > > > > > > > > > > > As I noted in the PR, the beta flag appears to be missing. > > Given > > > > this > > > > > > and > > > > > > > > several unresolved comments, it would have been better to > > address > > > > the > > > > > > > > feedback before merging. There is no rush, and considering > > > > community > > > > > > > > feedback is essential. > > > > > > > > > > > > > > > > While I am very excited about the semantics feature in > Polaris > > > and > > > > > > > > leveraging Apache Ossie, we must proceed with community > > consensus > > > > > > rather > > > > > > > > than rushing. > > > > > > > > > > > > > > > > I propose that we keep the PR merged but follow up > immediately > > to > > > > > > include > > > > > > > > the necessary beta wording and address the remaining > concerns. > > > > > > > > > > > > > > > > Regards, > > > > > > > > JB > > > > > > > > > > > > > > > > Le jeu. 2 juil. 2026 à 21:06, Yufei Gu <[email protected] > > > > a > > > > > écrit > > > > > > : > > > > > > > > > > > > > > > > > Thanks, Robert. I agree that we should clarify the beta > > wording > > > > and > > > > > > > > review > > > > > > > > > the payload shape before considering the API stable. > > > > > > > > > > > > > > > > > > That said, I'm not sure those items need to block a > release. > > > The > > > > > > > > question, > > > > > > > > > to me, is whether we're comfortable allowing a WIP feature > to > > > > > evolve > > > > > > > > across > > > > > > > > > multiple releases. My understanding is that Polaris > releases > > > are > > > > > not > > > > > > > tied > > > > > > > > > to any individual feature reaching completion. Instead, we > > > > release > > > > > > the > > > > > > > > > project as it stands, while features continue to mature > over > > > > time. > > > > > We > > > > > > > > > already have precedents for this. For example, Generic > Tables > > > > > shipped > > > > > > > as > > > > > > > > > experimental in 1.0.0 incubating and were only marked GA in > > > 1.3.0 > > > > > > > > > incubating, and many other features also fit to the same > > > pattern, > > > > > > like > > > > > > > > > event handling. > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 10:52 AM Robert Stupp < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Since the vote was cancelled and PR #4816 merged > > immediately > > > > > > > > afterward, I > > > > > > > > > > do not think the public record should be read as > consensus > > on > > > > the > > > > > > > exact > > > > > > > > > API > > > > > > > > > > shape. I may be missing a separate consensus signal, but > I > > > > think > > > > > we > > > > > > > > > should > > > > > > > > > > follow up before release with explicit beta/Ossie > wording, > > > > > > > > > > endpoint-discovery cleanup, and payload-shape review. > > > > > > > > > > > > > > > > > > > > Robert > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 7:27 AM Jean-Baptiste Onofré < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > > > > > > > I agree with your suggestion. We can begin the > > preparatory > > > > > steps > > > > > > to > > > > > > > > > host > > > > > > > > > > > Ossie models now and proceed with the integration as > soon > > > as > > > > > the > > > > > > > > Ossie > > > > > > > > > > > specification becomes available. > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 6:55 PM Yufei Gu < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > JB, waiting for Ossie makes sense to me. That said, > I'd > > > > > suggest > > > > > > > we > > > > > > > > > work > > > > > > > > > > > in > > > > > > > > > > > > parallel where possible. > > > > > > > > > > > > > > > > > > > > > > > > From my understanding, the main dependency on Ossie > is > > > the > > > > > JSON > > > > > > > > > > > validator, > > > > > > > > > > > > which will arrive a bit later. Most of the other work > > can > > > > > > proceed > > > > > > > > > > > > independently in the meantime. > > > > > > > > > > > > > > > > > > > > > > > > I'm also OK moving forward without a formal vote if > > > that's > > > > > the > > > > > > > > > > preferred > > > > > > > > > > > > approach. I would keep the API explicitly in **beta** > > and > > > > > make > > > > > > it > > > > > > > > > clear > > > > > > > > > > > > that we intend to transition to the Ossie > specification > > > > once > > > > > it > > > > > > > > > becomes > > > > > > > > > > > > available. That gives us a path to make progress now > > > while > > > > > > > > remaining > > > > > > > > > > > > aligned with the future direction. > > > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 5:38 AM Jean-Baptiste Onofré > < > > > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > It might be worth waiting for Apache Ossie to > > bootstrap > > > > > > before > > > > > > > > > > > > proceeding. > > > > > > > > > > > > > While I agree with the intent, I want to avoid > > > > duplicative > > > > > > work > > > > > > > > > where > > > > > > > > > > > we > > > > > > > > > > > > > implement the OSI spec first and the Apache Ossie > > spec > > > > as a > > > > > > > > > > follow-up. > > > > > > > > > > > > > Since the OSI spec is transitioning into the Ossie > > > spec, > > > > > and > > > > > > an > > > > > > > > > > initial > > > > > > > > > > > > > Apache Ossie spec release is expected soon, I > suggest > > > we > > > > > > pause > > > > > > > > this > > > > > > > > > > > vote. > > > > > > > > > > > > > > > > > > > > > > > > > > Additionally, I would prefer to find a consensus > > rather > > > > > than > > > > > > > > > holding > > > > > > > > > > a > > > > > > > > > > > > > vote, as voting is typically reserved for when > > > consensus > > > > is > > > > > > > > > difficult > > > > > > > > > > > to > > > > > > > > > > > > > reach. This vote could likely be considered a code > > > > > > modification > > > > > > > > > under > > > > > > > > > > > > > Apache voting guidelines ( > > > > > > > > > > > https://www.apache.org/foundation/voting.html > > > > > > > > > > > > ). > > > > > > > > > > > > > > > > > > > > > > > > > > My recommendation is to pause the vote for now and > > > > leverage > > > > > > > > Apache > > > > > > > > > > > Ossie, > > > > > > > > > > > > > which I am currently working on. > > > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 12:51 AM Yufei Gu < > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Beta sounds good to me. We can clarify the beta > > > status > > > > > in a > > > > > > > > > > follow-up > > > > > > > > > > > > doc > > > > > > > > > > > > > > change. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 29, 2026 at 3:27 PM Dmitri > > Bourlatchkov < > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to understand the impact of this vote > > > > better. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Are you proposing to accept this new API > without > > > > first > > > > > > > going > > > > > > > > > > > through > > > > > > > > > > > > a > > > > > > > > > > > > > > > "beta" phase? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the beta process is pretty standard for > > new > > > > > APIs > > > > > > in > > > > > > > > > > Polaris > > > > > > > > > > > > > these > > > > > > > > > > > > > > > days (e.g. the Metrics API). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 29, 2026 at 1:38 PM Yufei Gu < > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to call a vote on accepting the OSI > > > > Semantic > > > > > > > Model > > > > > > > > > API > > > > > > > > > > > > > > > > specification proposed in PR #4816( > > > > > > > > > > > > > > > > https://github.com/apache/polaris/pull/4816) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This proposal introduces the initial > > scaffolding > > > > for > > > > > > the > > > > > > > > Open > > > > > > > > > > > > > Semantic > > > > > > > > > > > > > > > > Interchange (OSI) Semantic Model API in > > Polaris. > > > > The > > > > > > > > primary > > > > > > > > > > goal > > > > > > > > > > > > is > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > establish the REST API surface and associated > > > > > > > > specification, > > > > > > > > > > > while > > > > > > > > > > > > > > > leaving > > > > > > > > > > > > > > > > the implementation intentionally minimal. > This > > > > > > provides a > > > > > > > > > > stable > > > > > > > > > > > > > > > foundation > > > > > > > > > > > > > > > > for future work while allowing the community > to > > > > > iterate > > > > > > > on > > > > > > > > > the > > > > > > > > > > > > > > > > implementation independently. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This vote will remain open for 72 hours. > Please > > > > vote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1 Accept the proposal > > > > > > > > > > > > > > > > [ ] 0 No opinion > > > > > > > > > > > > > > > > [ ] -1 Do not accept the proposal (please > > provide > > > > > > > > reasoning) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks to everyone who participated in the > > design > > > > > > > > discussions > > > > > > > > > > and > > > > > > > > > > > > > > > reviews. > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
