Hi all, Responding to Yufei's questions: I don't think we can enforce that users will use OTel. And that's fine - I think we should generate the Request ID in the case that a request ID was not defined in the header and OTel is not enabled. This may not be the thread for that, but we can discuss this separately.
I think we've reached a consensus on this thread between those who've engaged on this thread: we should only use Request ID to correlate events within the same request. Thanks Dmitri and Yufei! If others have thoughts, please feel free to add! Best, Adnan Hemani On Tue, Nov 4, 2025 at 11:09 AM Dmitri Bourlatchkov <[email protected]> wrote: > Sorry, I meant Adnan, of course :) > > On Tue, Nov 4, 2025 at 2:08 PM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi Prashant, > > > > Thanks for clarifying that pair-wise correlation of multiple before/after > > event pairs per request is _not_ required. > > > > That was my only concern in this thread. I do not have any additional use > > cases in mind. I was just trying to achieve clarity on what was actually > > required and what was collateral :) > > > > Cheers, > > Dmitri. > > > > > > On Mon, Nov 3, 2025 at 10:48 PM Adnan Hemani > > <[email protected]> wrote: > > > >> Hi Dmitri, > >> > >> Thanks for creating this thread! If what has sparked this thought was my > >> input earlier that users would like to correlate their events together, > >> I'd > >> like to make a clarification: I believe that users would like to > correlate > >> events coming from the same request to Polaris together. In which case, > I > >> think Request ID is sufficient to do what we need. If there is other > input > >> that we should be able to correlate pairs of Before/After events, then I > >> think we should discuss either adding "Correlation ID" (which I really > >> don't like, as it makes things quite confusing IMO) or putting a "Span > ID" > >> inside the `additionalProperties` of the event, like Yufei suggested > (and > >> we may have to generate that "Span ID"). > >> > >> As to Yufei's point, I don't think we need to explicitly store OTel > >> context > >> information. If it is present through Quarkus' integration with OTel, I > >> still believe the OTel Trace ID should be the Request ID, if the user > has > >> not defined a Request ID (sorry to mix the lines with the other threads' > >> topics on this). > >> > >> WDYT? > >> > >> Best, > >> Adnan Hemani > >> > >> On Fri, Oct 31, 2025 at 11:47 AM Yufei Gu <[email protected]> wrote: > >> > >> > Agreed with Dmitri. > >> > > >> > X-Request-Id (or Request-Id) is not a standard HTTP header but is > widely > >> > used across systems, including Polaris, for logging, events, and > >> metrics. > >> > Its primary purpose is debuggability. For example, it lets us > correlate > >> > logs and events for the same HTTP call. > >> > Example: > >> > Log: {timestamp: ..., request-id: 123, message: "Update table > >> > successfully!"} > >> > Event: {type: BeforeUpdateTable, request-id: 123}, {type: > >> AfterUpdateTable, > >> > request-id: 123} > >> > > >> > *Request ID* should remain the canonical identifier for every request > >> > handled by Polaris. > >> > > >> > - If missing, Polaris generates one (e.g., UUIDv7 or brings back > the > >> one > >> > Adnan built). > >> > - All logs, events, and metrics include it for consistent internal > >> and > >> > downstream correlation. No need to invent another concept like > >> > "correlated ID" for event only, which introduce inconsistency and > >> make > >> > correlation harder. > >> > > >> > *OTel context* is OPTIONAL. > >> > > >> > - If present, persist it alongside for tracing. Put it in > >> > `additionalProperties` for events if people need it. > >> > - If absent, users decide to start a new trace (nice to have) or > not. > >> > > >> > > >> > Yufei > >> > > >> > > >> > On Fri, Oct 31, 2025 at 10:37 AM Dmitri Bourlatchkov < > [email protected]> > >> > wrote: > >> > > >> > > Hi All, > >> > > > >> > > I'd like to broaden the events / OTel discussion [1][2] a bit. > >> Apologies > >> > > for forking the email threads even more, but I think this aspect > >> > deserves a > >> > > focused discussion. > >> > > > >> > > From previous messages my impression is that there are use cases for > >> > > strongly correlating "before" and "after" events. In that situation > >> the > >> > > full OTel context may be too volatile. For example, nothing > guarantees > >> > that > >> > > the "before" and "after" events are produced under the same OTel > span > >> in > >> > > Polaris. > >> > > > >> > > Another aspect of this is multiple before/after event pairs may be > >> > produced > >> > > under the same API request. Assuming event consumers do want to > >> > > re-establish correct event pairs, I believe using only the request > ID > >> is > >> > > not sufficient for that purpose. > >> > > > >> > > We probably need to restore the Polaris request ID generator but use > >> it > >> > to > >> > > produce an ID per group of events (before/after) and publish / > persist > >> > that > >> > > ID with the event data. > >> > > > >> > > WDYT? > >> > > > >> > > [1] > https://lists.apache.org/thread/bb1qyxjt827t3tomv2xp0s1kovxjsp94 > >> > > [2] > https://lists.apache.org/thread/5dpyo0nn2jbnjtkgv0rm1dz8mpt132j9 > >> > > > >> > > Thanks, > >> > > Dmitri. > >> > > > >> > > >> > > >
