Hi all,
As a follow-up to [1], I'm starting this thread to discuss how we can
persist client-generated request IDs and OTel context in our database.
Quoting myself [2], the requirements I think we want to fulfil are:
1. Only one correlation ID is enough
2. The correlation ID is an opaque string
3. The main use case is to find events with matching correlation IDs
4. The only query pattern is by exact match
A few options were suggested:
1) Two separate columns: request_id and otel_context, both of type
TEXT (nullable).
- Pros: Easy to implement and offers good read performance,
especially with indexes.
- Cons: Could be overkill, as often one context is sufficient for
correlating records.
2) A single column: (e.g., correlation_id, final name TBD) of type TEXT.
- Pros: Same as option 1.
- Cons: If both a request ID and OTel context are available, we
can only store one. There's no straightforward way to identify the
type of context stored (unless we use a prefix).
3) Two columns: correlation_id and correlation_id_type.
- Pros: Same as option 1, and it addresses the issue of
identifying the ID type.
- Cons: Might be over-engineered. Is the ID type truly essential?
Isn't it opaque?
4) Leverage the existing additional_properties column: (JSONB in
Postgres, TEXT in H2).
- Pros: Simple and flexible due to its schemaless nature, allowing
us to add anything we need.
- Cons: Query performance might not be optimal, though indexes could help.
What are your thoughts?
Thanks,
Alex
[1] https://lists.apache.org/thread/bb1qyxjt827t3tomv2xp0s1kovxjsp94
[2] https://lists.apache.org/thread/fqjjmxc6v8bbynwd5xfz83ngmp6gqqxj
