0AyanamiRei commented on issue #53798: URL: https://github.com/apache/doris/issues/53798#issuecomment-3131278327
**If anyone else would like to take over this issue, feel free to leave a comment**—I’ll move on to another issue first. Below are some helpful notes on the strategy for this issue: <img width="1851" height="916" alt="Image" src="https://github.com/user-attachments/assets/b5be9c50-8a27-4528-a5cf-65bf77da26ba"; /> The entire call chain for the BE-side warm-up is shown in the diagram. After the BE breaks down the job into segment-level tasks and submits them via `submit_download_task` to the `_task_queue`, the `_poller` checks this queue and assigns tasks to the `_workers` thread pool via `submit_func` to execute `download_blocks`. The `wait` (`CountdownEvent`) is propagated along the chain, and the BE then calls `wait->timed_wait(time)` to await task completion. To quickly respond to cancel messages from the FE, I think modifications can be made at two key points 1. First proposed modification : Split `WAIT_TIME_SECONDS` into multiple slots to allow responsiveness to external interruptions (e.g., cancel requests). This reduces latency compared to checking for cancellation only after processing an entire batch, though a maximum wait time still exists (equal to the slot duration). 2. Second proposed modification : Extend `FileCacheBlockDownloader` to provide not only `submit_download_task` but also a `cancel_download_task` interface, notifying download threads to invoke a callback after each subtask completes. The worst-case overhead here would be: max_overhead = time_to_submit_entire_job + time_to_load_one_config::s3_write_buffer_size_chunk. Maybe you will think the second solution is more elegant, but you'll need to address how to maintain a record of a job's `_task_queue `rather than just popping tasks and handing them over to worker threads. You may need to discuss with the relevant stakeholders to determine which approach is better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
