Hi folks, I believe the "façade" Semantic REST API should be the primary focus here, as it represents the user-facing layer. The internal representation is less of a concern since it can be updated or changed as needed.
We should align on the API surface defined in the design document: https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.t6dop8lqx3uq My takeaways from this discussion are: 1. There seems to be no concern with introducing the new endpoints based on /v1/{prefix}/namespaces/{ns}/semantic-models. 2. The discussion is primarily centered on using "raw" JSON in the endpoints. Personally, I think using raw JSON is acceptable because we can translate it to our internal representation. However, we should consider that this may introduce server-side overhead regarding marshalling/unmarshalling and memory pressure. To reach a consensus, I suggest starting a fresh discussion thread specifically focused on the REST API and the usage of JSON payloads. Yufei, would you mind starting that thread? Thoughts? Regards, JB On Wed, Jul 8, 2026 at 3:53 AM Yufei Gu <[email protected]> wrote: > Thanks for the feedback. This is a good discussion. > > I'm fine with either representing the Ossie payload as a raw string or as > an opaque JSON object in the REST spec. However, bringing the full Ossie > JSON structure into the REST API is problematic because the Ossie schema is > versioned. Doing so would tightly couple the REST spec to Ossie versions. I > agree with Adam that keeping the payload opaque allows us to support > different semantic model schemas without changing the REST API. We would > only need to update the validator as the Ossie spec evolves. > > With this approach, whenever Ossie introduces a new version, Polaris > doesn't need to change the REST spec. We would simply add support for the > new version in the validator. > > I'm not sure I fully understand the client-side concern. Clients will > follow the REST API regardless of whether the feature is in beta. In fact, > I think the beta status gives us more flexibility, since breaking changes > are acceptable while the API is still evolving. > > Just in case it helps, the design doc [1] explains this: > > > Round trip is general byte-preserving, subject to JSON's key ordering and > > whitespace flexibility, and semantically lossless. Every field, including > > custom_extensions, survives write to read. > > [1] > > https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.a2w9eycpqhof > > > Yufei > > > On Tue, Jul 7, 2026 at 7:57 AM Jean-Baptiste Onofré <[email protected]> > wrote: > > > Hi > > > > I think we have to distinguish two aspects here: > > 1. how do we "store" the semantic models in the Polaris service (json > raw, > > variant, object, ...). I'm not much concerned about that as it's our > > internal plumbing. > > 2. how the client will interact the semantic models, e.g. the façade. > Here, > > we must be cautious and reach a consensus. > > > > If I agree we can move forward on 1 without much concern (we can always > > debate and change the internal representation), I suggest focusing on > (2). > > My preference would be to façade the Ossie JSON as an "object" (not as a > > raw string). It gives us the flexibility to change the "mapping" between > > the client request/response and our inner storage. > > The API can manage the Ossie "objects" (create/update/resolve, e.g. > > CRUD++). > > > > I suggest to focus on this part now, as it's really fundamental on the > user > > facing part. > > > > Regards > > JB > > > > On Tue, Jul 7, 2026 at 4:41 PM Robert Stupp <[email protected]> wrote: > > > > > For beta, I think the minimum contract should be that the API makes the > > > intended client model explicit. > > > > > > If `semantic_model` stays a string, I think the docs/spec should say > that > > > clients should treat it as opaque round-trip document content, not as a > > > generated-client-modeled Ossie object. > > > In that case, I also think we should define at least basic operational > > > bounds, for example, the maximum accepted document size and validation > > > expectations, because semantic documents can grow beyond small > examples. > > > > > > If we expect clients to inspect or build UX around the payload through > > > generated clients, I would prefer modeling the Ossie document as JSON > > > rather than escaped JSON in a string. > > > > > > Regarding discovery, I think beta should also state whether v1 is > > > intentionally limited to namespace/name CRUD, or whether clients should > > > expect to discover usable models by version/type/capability later. > > > I do not think we need to solve rich search now, but we should avoid > > > implying more discoverability than the API provides. > > > > > > I agree this does not need to stop all semantic-model work, and I do > not > > > think it should block the beta wording cleanup in #4983. > > > > > > My main concern is that these points should be resolved before #4961 > > merges > > > as the first working CRUD API, because that is when client expectations > > > start to become sticky. > > > > > > On Tue, Jul 7, 2026 at 4:06 PM Adam Christian < > > > [email protected]> wrote: > > > > > > > Thanks, Robert! > > > > > > > > Here are my thoughts: > > > > 1. I'm fine with either making the REST API accept just the JSON > string > > > or > > > > modeling it like an Ossie document. We just need to make a call. > > > > - For the JSON string pros, we will be able to accept different types > > of > > > > semantic model documents with different schemas without having to > > update > > > > the REST API. We would only have to update the validator. > > > > - For the modeling pros, we will be more prescriptive about what we > > > support > > > > and what we do not inside the REST API. It would also decrease the > > > > validator's workload. > > > > For me, I could go either way. Do you have a preference? > > > > > > > > 2. I'll align the Semantic Model work with the registry endpoint. I > can > > > do > > > > that now. I don't think that this improvement needs to slow the > > Semantic > > > > Model work. I'll complete that in parallel. > > > > > > > > 3. If you look into the Semantic Model design document, there is a > > small > > > > section about the OSI Version Evolution [1]. It doesn't lay out > exactly > > > > what you are asking for, but it states that we will support multiple > > > > versions of the OSI documents. You can save 0.1.1 or 0.2.0. Now, I > > assume > > > > we will support different versions of the specification and perform > > light > > > > validation on them. We probably will want to document a support > matrix > > > with > > > > each version of Polaris as well. I don't think that this needs to > stop > > > > #4961, but I do think that we will need to speak about this before we > > > ship > > > > it out to beta. Do you have any expectations that must be met before > we > > > > ship? > > > > > > > > [1] - > > > > > > > > > > > > > > https://docs.google.com/document/d/1ZdI-1w_5LbyCMhvUhLCtOt-N1Z89L2P-oiGLaYayCZg/edit?tab=t.0#heading=h.n36d87je014l > > > > > > > > Go community, > > > > > > > > Adam > > > > > > > > On Tue, Jul 7, 2026 at 9:37 AM Robert Stupp <[email protected]> wrote: > > > > > > > > > Thanks for the beta/Ossie cleanup in #4983. I think that is useful > > and > > > > > should move forward. > > > > > > > > > > I do not think that fully closes the API-shape follow-up from > #4816, > > > > > though. Before #4961 turns the semantic-model API into real CRUD > > > > behavior, > > > > > I think we should still settle two things explicitly: > > > > > > > > > > 1. whether `semantic_model` should remain escaped JSON in a string, > > or > > > > > whether the API should model the Ossie document as JSON; > > > > > 2. whether endpoint advertisement should wait for / align with the > > > > endpoint > > > > > registry work in #4926, so semantic-model discovery does not > > hard-code > > > > > another optional non-IRC API path. > > > > > 3. how clients are expected to use the API in practice while it is > > > beta: > > > > > whether clients should treat the document as opaque round-trip > > storage, > > > > > whether Polaris intends generated clients to model the Ossie > payload, > > > and > > > > > what compatibility expectations clients should have before the API > > > > becomes > > > > > stable. > > > > > > > > > > I support keeping semantic models as beta/preparatory Ossie > document > > > > > hosting. I just want to avoid making the beta API sticky before the > > > > payload > > > > > and discovery shape are reviewed. > > > > > > > > > > On Mon, Jul 6, 2026 at 8:09 PM Yufei Gu <[email protected]> > > wrote: > > > > > > > > > > > Hi folks, here is the PR to add the beta wording. Please take a > > look, > > > > > > https://github.com/apache/polaris/pull/4983. > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 11:53 PM Jean-Baptiste Onofré < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Yufei, Robert, > > > > > > > > > > > > > > I agree regarding the release process; shipping beta features > is > > > > > > consistent > > > > > > > with our past practices. > > > > > > > > > > > > > > As I noted in the PR, the beta flag appears to be missing. > Given > > > this > > > > > and > > > > > > > several unresolved comments, it would have been better to > address > > > the > > > > > > > feedback before merging. There is no rush, and considering > > > community > > > > > > > feedback is essential. > > > > > > > > > > > > > > While I am very excited about the semantics feature in Polaris > > and > > > > > > > leveraging Apache Ossie, we must proceed with community > consensus > > > > > rather > > > > > > > than rushing. > > > > > > > > > > > > > > I propose that we keep the PR merged but follow up immediately > to > > > > > include > > > > > > > the necessary beta wording and address the remaining concerns. > > > > > > > > > > > > > > Regards, > > > > > > > JB > > > > > > > > > > > > > > Le jeu. 2 juil. 2026 à 21:06, Yufei Gu <[email protected]> > a > > > > écrit > > > > > : > > > > > > > > > > > > > > > Thanks, Robert. I agree that we should clarify the beta > wording > > > and > > > > > > > review > > > > > > > > the payload shape before considering the API stable. > > > > > > > > > > > > > > > > That said, I'm not sure those items need to block a release. > > The > > > > > > > question, > > > > > > > > to me, is whether we're comfortable allowing a WIP feature to > > > > evolve > > > > > > > across > > > > > > > > multiple releases. My understanding is that Polaris releases > > are > > > > not > > > > > > tied > > > > > > > > to any individual feature reaching completion. Instead, we > > > release > > > > > the > > > > > > > > project as it stands, while features continue to mature over > > > time. > > > > We > > > > > > > > already have precedents for this. For example, Generic Tables > > > > shipped > > > > > > as > > > > > > > > experimental in 1.0.0 incubating and were only marked GA in > > 1.3.0 > > > > > > > > incubating, and many other features also fit to the same > > pattern, > > > > > like > > > > > > > > event handling. > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 10:52 AM Robert Stupp <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > Since the vote was cancelled and PR #4816 merged > immediately > > > > > > > afterward, I > > > > > > > > > do not think the public record should be read as consensus > on > > > the > > > > > > exact > > > > > > > > API > > > > > > > > > shape. I may be missing a separate consensus signal, but I > > > think > > > > we > > > > > > > > should > > > > > > > > > follow up before release with explicit beta/Ossie wording, > > > > > > > > > endpoint-discovery cleanup, and payload-shape review. > > > > > > > > > > > > > > > > > > Robert > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2026 at 7:27 AM Jean-Baptiste Onofré < > > > > > [email protected] > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > > > > > I agree with your suggestion. We can begin the > preparatory > > > > steps > > > > > to > > > > > > > > host > > > > > > > > > > Ossie models now and proceed with the integration as soon > > as > > > > the > > > > > > > Ossie > > > > > > > > > > specification becomes available. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 6:55 PM Yufei Gu < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > JB, waiting for Ossie makes sense to me. That said, I'd > > > > suggest > > > > > > we > > > > > > > > work > > > > > > > > > > in > > > > > > > > > > > parallel where possible. > > > > > > > > > > > > > > > > > > > > > > From my understanding, the main dependency on Ossie is > > the > > > > JSON > > > > > > > > > > validator, > > > > > > > > > > > which will arrive a bit later. Most of the other work > can > > > > > proceed > > > > > > > > > > > independently in the meantime. > > > > > > > > > > > > > > > > > > > > > > I'm also OK moving forward without a formal vote if > > that's > > > > the > > > > > > > > > preferred > > > > > > > > > > > approach. I would keep the API explicitly in **beta** > and > > > > make > > > > > it > > > > > > > > clear > > > > > > > > > > > that we intend to transition to the Ossie specification > > > once > > > > it > > > > > > > > becomes > > > > > > > > > > > available. That gives us a path to make progress now > > while > > > > > > > remaining > > > > > > > > > > > aligned with the future direction. > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 5:38 AM Jean-Baptiste Onofré < > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > It might be worth waiting for Apache Ossie to > bootstrap > > > > > before > > > > > > > > > > > proceeding. > > > > > > > > > > > > While I agree with the intent, I want to avoid > > > duplicative > > > > > work > > > > > > > > where > > > > > > > > > > we > > > > > > > > > > > > implement the OSI spec first and the Apache Ossie > spec > > > as a > > > > > > > > > follow-up. > > > > > > > > > > > > Since the OSI spec is transitioning into the Ossie > > spec, > > > > and > > > > > an > > > > > > > > > initial > > > > > > > > > > > > Apache Ossie spec release is expected soon, I suggest > > we > > > > > pause > > > > > > > this > > > > > > > > > > vote. > > > > > > > > > > > > > > > > > > > > > > > > Additionally, I would prefer to find a consensus > rather > > > > than > > > > > > > > holding > > > > > > > > > a > > > > > > > > > > > > vote, as voting is typically reserved for when > > consensus > > > is > > > > > > > > difficult > > > > > > > > > > to > > > > > > > > > > > > reach. This vote could likely be considered a code > > > > > modification > > > > > > > > under > > > > > > > > > > > > Apache voting guidelines ( > > > > > > > > > > https://www.apache.org/foundation/voting.html > > > > > > > > > > > ). > > > > > > > > > > > > > > > > > > > > > > > > My recommendation is to pause the vote for now and > > > leverage > > > > > > > Apache > > > > > > > > > > Ossie, > > > > > > > > > > > > which I am currently working on. > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > JB > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 30, 2026 at 12:51 AM Yufei Gu < > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Beta sounds good to me. We can clarify the beta > > status > > > > in a > > > > > > > > > follow-up > > > > > > > > > > > doc > > > > > > > > > > > > > change. > > > > > > > > > > > > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 29, 2026 at 3:27 PM Dmitri > Bourlatchkov < > > > > > > > > > > [email protected]> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Yufei, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to understand the impact of this vote > > > better. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Are you proposing to accept this new API without > > > first > > > > > > going > > > > > > > > > > through > > > > > > > > > > > a > > > > > > > > > > > > > > "beta" phase? > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the beta process is pretty standard for > new > > > > APIs > > > > > in > > > > > > > > > Polaris > > > > > > > > > > > > these > > > > > > > > > > > > > > days (e.g. the Metrics API). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 29, 2026 at 1:38 PM Yufei Gu < > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to call a vote on accepting the OSI > > > Semantic > > > > > > Model > > > > > > > > API > > > > > > > > > > > > > > > specification proposed in PR #4816( > > > > > > > > > > > > > > > https://github.com/apache/polaris/pull/4816) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This proposal introduces the initial > scaffolding > > > for > > > > > the > > > > > > > Open > > > > > > > > > > > > Semantic > > > > > > > > > > > > > > > Interchange (OSI) Semantic Model API in > Polaris. > > > The > > > > > > > primary > > > > > > > > > goal > > > > > > > > > > > is > > > > > > > > > > > > to > > > > > > > > > > > > > > > establish the REST API surface and associated > > > > > > > specification, > > > > > > > > > > while > > > > > > > > > > > > > > leaving > > > > > > > > > > > > > > > the implementation intentionally minimal. This > > > > > provides a > > > > > > > > > stable > > > > > > > > > > > > > > foundation > > > > > > > > > > > > > > > for future work while allowing the community to > > > > iterate > > > > > > on > > > > > > > > the > > > > > > > > > > > > > > > implementation independently. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This vote will remain open for 72 hours. Please > > > vote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1 Accept the proposal > > > > > > > > > > > > > > > [ ] 0 No opinion > > > > > > > > > > > > > > > [ ] -1 Do not accept the proposal (please > provide > > > > > > > reasoning) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks to everyone who participated in the > design > > > > > > > discussions > > > > > > > > > and > > > > > > > > > > > > > > reviews. > > > > > > > > > > > > > > > Yufei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
