Hi Dmitri, Thanks for creating this thread! If what has sparked this thought was my input earlier that users would like to correlate their events together, I'd like to make a clarification: I believe that users would like to correlate events coming from the same request to Polaris together. In which case, I think Request ID is sufficient to do what we need. If there is other input that we should be able to correlate pairs of Before/After events, then I think we should discuss either adding "Correlation ID" (which I really don't like, as it makes things quite confusing IMO) or putting a "Span ID" inside the `additionalProperties` of the event, like Yufei suggested (and we may have to generate that "Span ID").
As to Yufei's point, I don't think we need to explicitly store OTel context information. If it is present through Quarkus' integration with OTel, I still believe the OTel Trace ID should be the Request ID, if the user has not defined a Request ID (sorry to mix the lines with the other threads' topics on this). WDYT? Best, Adnan Hemani On Fri, Oct 31, 2025 at 11:47 AM Yufei Gu <[email protected]> wrote: > Agreed with Dmitri. > > X-Request-Id (or Request-Id) is not a standard HTTP header but is widely > used across systems, including Polaris, for logging, events, and metrics. > Its primary purpose is debuggability. For example, it lets us correlate > logs and events for the same HTTP call. > Example: > Log: {timestamp: ..., request-id: 123, message: "Update table > successfully!"} > Event: {type: BeforeUpdateTable, request-id: 123}, {type: AfterUpdateTable, > request-id: 123} > > *Request ID* should remain the canonical identifier for every request > handled by Polaris. > > - If missing, Polaris generates one (e.g., UUIDv7 or brings back the one > Adnan built). > - All logs, events, and metrics include it for consistent internal and > downstream correlation. No need to invent another concept like > "correlated ID" for event only, which introduce inconsistency and make > correlation harder. > > *OTel context* is OPTIONAL. > > - If present, persist it alongside for tracing. Put it in > `additionalProperties` for events if people need it. > - If absent, users decide to start a new trace (nice to have) or not. > > > Yufei > > > On Fri, Oct 31, 2025 at 10:37 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi All, > > > > I'd like to broaden the events / OTel discussion [1][2] a bit. Apologies > > for forking the email threads even more, but I think this aspect > deserves a > > focused discussion. > > > > From previous messages my impression is that there are use cases for > > strongly correlating "before" and "after" events. In that situation the > > full OTel context may be too volatile. For example, nothing guarantees > that > > the "before" and "after" events are produced under the same OTel span in > > Polaris. > > > > Another aspect of this is multiple before/after event pairs may be > produced > > under the same API request. Assuming event consumers do want to > > re-establish correct event pairs, I believe using only the request ID is > > not sufficient for that purpose. > > > > We probably need to restore the Polaris request ID generator but use it > to > > produce an ID per group of events (before/after) and publish / persist > that > > ID with the event data. > > > > WDYT? > > > > [1] https://lists.apache.org/thread/bb1qyxjt827t3tomv2xp0s1kovxjsp94 > > [2] https://lists.apache.org/thread/5dpyo0nn2jbnjtkgv0rm1dz8mpt132j9 > > > > Thanks, > > Dmitri. > > >
