Hello Aurorans, Please enlighten me.
I think job history is a critical feature for Aurora. A. Do you agree? B. Is this feature secretly already in Aurora? C. If not, is this on your roadmap? D. Would you be interested in a patch or patches that adds job history to Aurora? Below I discuss why I think this is an important feature and some thoughts on an implementation. Job history has a number of uses: 1. Debugging production issues after the job has been updated. I may need to know the exact configuration of a system at a previous point in time in order to debug an issue. 2. Rolling back to a previous job configuration after a bad release. How I think Aurora works: As far as I can tell from the Aurora source, job history is discarded. The MemJobStore replaces Job entries when a job is updated, so you lose the old Job configuration. The log is truncated every time a Snapshot is taken and the snapshots do not contain job history. This seems like a sound decision given that the job history will grow forever, but means there's no history we can really audit. How job history might work: Instead of building job history into the scheduler one might write an independent process that consumed the logs generated by the scheduler and built up a database of job history information. It would then provide a REST interface for querying the job history. This would keep the scheduler free from dealing with job history. Any feedback is appreciated. Thanks. -David Siegel
