abhishekrb19 commented on PR #19191: URL: https://github.com/apache/druid/pull/19191#issuecomment-4238614607
> I was debating the same, but felt that this doesn't really fit into the existing StreamSupervisor implementation for these reasons: > > * taskDuration doesn't make sense in the context of a backfill task since ideally we would just want it to run to completion > * Supervisors are long running. With this kind of short-lived Supervisor I think this requires some additional logic in the SupervisorManager to track the tasks to completion and then terminate the Supervisor. > * These differences make me think it'd be worthwhile to implement a new kind of Supervisor that can handle task tracking and handle termination. > >Wouldn't it be better to just directly use startSequenceNumbers/endSequenceNumbers and set partitionSequenceNumberMap like in this PR These are good points. I wonder if there’s any reason not to expose start and end sequence numbers in the supervisor config. Doing so would let the supervisor transition to a terminal state once those values are explicitly set and the specified offset range has been successfully read and published. Also tagging @gianm @jtuglu1 @kfaraz - curious if you have any thoughts here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
