Re: Removal of CallContext.copy()

2025-08-14 Thread Dmitri Bourlatchkov
Hi Dennis, This may be quite off topic here, so I'll keep it short. I'm open to discussing this matter in a dedicated thread :) My reading is that CallContext is effectively an alternative to CDI. However, CDI is richer in functionality. Not all requests / calls have the same set of contextual da

Re: Removal of CallContext.copy()

2025-08-14 Thread Yufei Gu
To summarize a bit: I agreed that the task context should be treated differently from both call context and realm context. We should design the task context explicitly, and it will inevitably need to be serialized so that other nodes can execute the task with enough context. When designing it, we

Re: Removal of CallContext.copy()

2025-08-14 Thread Robert Stupp
I think the discussion has deviated quite a bit. For posterity, the only use case in Polaris for the only call site of `.copy()` is "purge table data". All that needs is the information about "what shall be purged" and during execution a way to get access to the object storage. Any task execution

Re: Removal of CallContext.copy()

2025-08-13 Thread Dennis Huo
I feel like we've discussed this before but I can't find the previous threads. The purpose of the CallContext is explicitly to encapsulate the set of information needed to represent running internal operations on behalf of an abstract notion of a "request", precisely to *avoid* hiding it all in CDI

Re: Removal of CallContext.copy()

2025-08-13 Thread yun zou
Hi, While in theory, not all data within CallContext or RealmContext is required to execute the task, t he current TaskExecutor implementation does rely on these objects during execution. For instance, consider the code in TaskExecutorImpl

Re: Removal of CallContext.copy()

2025-08-13 Thread Dmitri Bourlatchkov
Hi Yufei, I would oppose serializing CallContext (especially across server nodes). Execution context is established based on a request. Executing a Task is one of possible requests. As such it has well defined parameters and begin/end boundaries. Any context that a task needs should naturally flo

Re: Removal of CallContext.copy()

2025-08-13 Thread Yufei Gu
If we want tasks to run on any node, we can’t avoid serializing the call context, or part of it like the realm context. The refactor, as written, could NOT support tasks on any node as well. As long as we add a field other than the realm ID to the realm context, there is no way to construct a realm

Re: Removal of CallContext.copy()

2025-08-13 Thread Eric Maynard
I agree that we shouldn't need the entire CallContext in each & every task -- tasks should include whatever information is necessary to execute them, and in the end CallContext.copy() shouldn't be needed. However, at least in Will's proposal, we would keep the existing framework around (with its u

Re: Removal of CallContext.copy()

2025-08-13 Thread Robert Stupp
Thanks for the reply Yufei, but the intent of all tasks-proposals is to be able to execute tasks on _any_ node. Are you suggesting making (Polaris)CallContext serializable? On Wed, Aug 13, 2025 at 3:12 AM Yufei Gu wrote: > > > To still let TaskExecutorImpl making "safe clones", a functionality to

Re: Removal of CallContext.copy()

2025-08-12 Thread Yufei Gu
> To still let TaskExecutorImpl making "safe clones", a functionality to get (fresh) instances of RealmContext is required. To enable this, the RealmContextResolver has been enhanced with "RealmContext lookups" by realm-ID. That in turn led to splitting the HTTP/REST-to-realm-context resolution int

Removal of CallContext.copy()

2025-08-12 Thread Robert Stupp
Hi all, quick heads up that there's a PR to remove CallContext.copy(), which is only used from tasks. This change is also part of the effort to have async & reliable tasks running "anywhere". Robert [1] https://github.com/apache/polaris/pull/2294