yuqi1129 opened a new issue, #13043: URL: https://github.com/apache/gravitino/issues/13043
### Version main branch ### Describe what's wrong `JobManager.cleanUpStagingDirs()` (`core/src/main/java/org/apache/gravitino/job/JobManager.java:764-800`) reaps finished jobs by calling `entityStore.delete(...)` directly: ```java entityStore.delete(NameIdentifierUtil.ofJob(metalake, job.name()), Entity.EntityType.JOB); ``` `JobManager` sits at the bottom of the dispatcher chain assembled in `GravitinoEnv` (`JobHookDispatcher` → `JobEventDispatcher(eventBus)` → `JobTemplateValidationDispatcher` → `JobManager`, see `GravitinoEnv.java:948-952`), so this deletion never passes through `JobEventDispatcher` and no listener observes it. There is also no `DeleteJobEvent` in `core/src/main/java/org/apache/gravitino/listener/api/event/job/` at all: job templates have `Register`, `Alter` and `Delete` events, while jobs only have `Run`, `Cancel`, `Get` and `List`. Any listener that maintains a projection of jobs therefore keeps serving jobs that no longer exist, and its store grows without bound, because the only deletion path is invisible to it. ### Error message and/or stacktrace None. The deletion is silent by construction, which is the bug. ### How to reproduce 1. Register an event listener that handles `JobEvent` and logs every event it receives. 2. Set `gravitino.job.stagingDirKeepTimeInMs` to a small value so the reaper runs promptly. 3. Run a job and let it reach `SUCCEEDED`. 4. Wait for `cleanUpStagingDirs()` to delete it, and confirm with `listJobs` that it is gone. 5. The listener receives `RunJobEvent` but never any deletion event, so a projection built from these events still contains the job. ### Additional context Found while adding a searchable projection of jobs downstream. Suggested fix: add `DeleteJobEvent` (with the matching `PreEvent` and `FailureEvent`, for consistency with the job template events) and give the reaper a way to emit it. Either pass the event bus into `JobManager`, or expose a deletion callback that `GravitinoEnv` wires to the event bus; the second keeps `JobManager` free of a listener dependency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
