Hi Gabor,
Thanks for clarifying! It's important that we share the same vision.
Best,
Zakelly
On Mon, Aug 26, 2024 at 3:56 PM Gabor Somogyi
wrote:
> Hi All,
>
> Thanks for the contributions!
> The asked umbrella document helped to resolve all the misunderstandings and
> align to a common resul
Hi All,
Thanks for the contributions!
The asked umbrella document helped to resolve all the misunderstandings and
align to a common result.
As a result I think we're ready for the voting thread.
BR,
G
On Mon, Aug 19, 2024 at 8:47 AM Gabor Somogyi
wrote:
> Hi All,
>
> Based on our agreement I
Hi All,
Based on our agreement I've created a draft Flink state observability
umbrella [1].
Please share your comments. It contains some details to give some insights
but the focus would be on the direction.
[1]
https://docs.google.com/document/d/1Du1-TShoOjaNDCahs3sgLWIpYkXzJPdSkgHcWLpELyw/edit
Hi Gabor,
I apologize for any confusion. Let me clarify my position.
The concept of state observability is important for users, and the current
FLIP seems to be a step in the right direction. However, before we proceed,
I suggest we discuss the final presentation of the state observability to
the
Hi David,
Thanks for sharing your thoughts!
> It sounds like you might already have an end-to-end solution in mind. It
would be really helpful if you could put that into writing so we can all
align our thinking.
It makes sense to create a high level vision.
> I’m not a fan of the mindset of “th
Hi Gabor,
Thanks for taking the initiative on this. It’s clear that significant
improvements are needed in this area, and parsing state files can be
incredibly challenging, even for those who are well-versed in it.
> Just to make it crystal clear, I’m not shooting for an ad-hoc tiny fix
but start
Hi Zakelly,
> I'd suggest we could think of this as a whole
In general I think we have the same idea in our mind about considering the
state observability as a whole, just we need to agree about the physical
task scheduling.
> But such a solution requires more design and discussion
I can't even
Hi Márton and Gabor,
Thanks for sharing context!
Yes, I'd admit that users need a more friendly way to explore states. And
it seems Flink lacks something like the state metadata store. I'd suggest
we could think of this as a whole, to store enough information for
querying, including operator name
Hi Zakelly,
Thanks for the feedback, let me elaborate on this.
In short Databricks has created a much more user friendly solution[1] for
state observability (based on Flink's state processor API) than what we
have now.
Up until now our state processor API was good enough but now we're lagging
be
Hi team,
Thank you for the proposal, Gabor and the review, Zakelly. Let me provide
more context on the intended outcome, let us agree on the desirability of
that first and then we can come back to the technical details.
The pain point that we observed with multiple users is that their
operations
Hi Gabor,
Thanks for the proposal! However, I find it a little strange. Are you
saying the user can set the operator uid but then doesn't know what they
set when debugging? Otherwise, is the `OperatorIdentifier.forUid("my-uid")`
feasible? I understand your point about potential cross-team work, bu
11 matches
Mail list logo
