vvcephei commented on a change in pull request #9262:
URL: https://github.com/apache/kafka/pull/9262#discussion_r491085819
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java
##########
@@ -292,7 +291,7 @@ public synchronized void clean() {
}
} catch (final IOException e) {
log.error("{} Failed to delete global state directory of {} due to
an unexpected exception",
- appId, logPrefix(), e);
+ logPrefix(), appId, e);
Review comment:
Just noticed something else wrong with this message. `e` will get
ignored. We should instead use `log.error(String.format("...", logPrefix(),
appId), e)`
##########
File path:
streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java
##########
@@ -306,59 +305,69 @@ public synchronized void clean() {
*/
public synchronized void cleanRemovedTasks(final long cleanupDelayMs) {
try {
- cleanRemovedTasks(cleanupDelayMs, false);
+ cleanRemovedTasksCalledByCleanerThread(cleanupDelayMs);
} catch (final Exception cannotHappen) {
throw new IllegalStateException("Should have swallowed
exception.", cannotHappen);
}
}
- private synchronized void cleanRemovedTasks(final long cleanupDelayMs,
- final boolean manualUserCall)
throws Exception {
- final File[] taskDirs = listAllTaskDirectories();
- if (taskDirs == null || taskDirs.length == 0) {
- return; // nothing to do
- }
-
- for (final File taskDir : taskDirs) {
+ private void cleanRemovedTasksCalledByCleanerThread(final long
cleanupDelayMs) {
+ for (final File taskDir : listAllTaskDirectories()) {
final String dirName = taskDir.getName();
final TaskId id = TaskId.parse(dirName);
if (!locks.containsKey(id)) {
- Exception exception = null;
try {
if (lock(id)) {
final long now = time.milliseconds();
final long lastModifiedMs = taskDir.lastModified();
if (now > lastModifiedMs + cleanupDelayMs) {
log.info("{} Deleting obsolete state directory {}
for task {} as {}ms has elapsed (cleanup delay is {}ms).",
logPrefix(), dirName, id, now -
lastModifiedMs, cleanupDelayMs);
-
- Utils.delete(taskDir,
Collections.singletonList(new File(taskDir, LOCK_FILE_NAME)));
- } else if (manualUserCall) {
- log.info("{} Deleting state directory {} for task
{} as user calling cleanup.",
- logPrefix(), dirName, id);
-
Utils.delete(taskDir,
Collections.singletonList(new File(taskDir, LOCK_FILE_NAME)));
}
}
- } catch (final OverlappingFileLockException | IOException e) {
- exception = e;
+ } catch (final OverlappingFileLockException | IOException
exception) {
+ log.warn("{} Swallowed the following exception during
deletion of obsolete state directory {} for task {}: {}",
+ logPrefix(), dirName, id, exception);
Review comment:
This log will be incomplete. We report the exception as the cause:
```suggestion
log.warn(String.format("%s Swallowed the following
exception during deletion of obsolete state directory %s for task %s",
logPrefix(), dirName, id), exception);
```
This feedback applies to pretty much all the warn/err logs in this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
