sbp opened a new pull request, #1451: URL: https://github.com/apache/tooling-trusted-releases/pull/1451
This is to prepare for further task work for #1450, but the idea of this PR is to make restarting tasks safer. We can have a situation where stale task workers (e.g. after a server restart; this happens a lot in dev) still exist and work is being done in parallel. If the stale task worker takes longer than the new one, it can overwrite the status etc. in the database. This PR prevents it from doing that, as each task worker gets the current execution generation of the task, and the generation counter is atomically incremented. Ideally we'd just terminate the old worker processes, but after a restart we have no idea whether the PIDs are of our workers or not. They might have been reallocated. One solution I was thinking about was to have a communication protocol, so that they could be checked directly, but then workers would need some kind of concurrent interrupt processor, and I figured that would be more complicated than it's worth. We do still have pessimistic stale task detection, and I've been thinking about how to improve it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
