Hi, Our team has been using Apache Kafka for over a year. Recently, we plan to replace the Kafka log directory with a symbolic link. For example, in the server.properties:
`log.dirs=/some/path/to/logdir` whereas `/some/path/to/logdir` is a symbolic link to `/actual/path/to/logdir`. To provide more context, the two paths reside in different disk partitions. However, we keep on failing to startup Kafka. Here's the error message: ``` [2024-12-30 18:14:52,434] INFO [ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [2024-12-30 18:14:52,444] INFO [KafkaServer id=MYID] Rewriting /path/to/logdir/meta.properties (kafka.server.KafkaServer) [2024-12-30 18:14:52,445] ERROR Error while writing meta.properties to /path/to/logdir (org.apache.kafka.storage.internals.log.LogDirFailureChannel) java.nio.file.FileAlreadyExistsException: /path/to/logdir at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:94) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397) at java.base/java.nio.file.Files.createDirectory(Files.java:700) at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807) at java.base/java.nio.file.Files.createDirectories(Files.java:753) at kafka.server.KafkaServer.$anonfun$startup$6(KafkaServer.scala:297) at org.apache.kafka.metadata.properties.MetaPropertiesEnsemble$Copier.writeLogDirChanges(MetaPropertiesEnsemble.java:373) at kafka.server.KafkaServer.startup(KafkaServer.scala:302) at kafka.Kafka$.main(Kafka.scala:112) at kafka.Kafka.main(Kafka.scala) [2024-12-30 18:14:52,466] ERROR Failed to create or validate data directory /path/to/logdir (org.apache.kafka.storage.internals.log.LogDirFailureChannel) java.io.IOException: Failed to load /path/to/logdir during broker startup at kafka.log.LogManager.$anonfun$createAndValidateLogDirs$1(LogManager.scala:181) 30-12-2024 10:14:52.468: at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:576) at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:574) at scala.collection.AbstractIterable.foreach(Iterable.scala:933) at kafka.log.LogManager.createAndValidateLogDirs(LogManager.scala:178) at kafka.log.LogManager.<init>(LogManager.scala:105) at kafka.log.LogManager$.apply(LogManager.scala:1539) at kafka.server.KafkaServer.startup(KafkaServer.scala:316) at kafka.Kafka$.main(Kafka.scala:112) at kafka.Kafka.main(Kafka.scala) [2024-12-30 18:14:52,469] ERROR Shutdown broker because none of the specified log dirs from /path/to/logdir can be created or validated (kafka.log.LogManager) ``` Environment - Kafka version: 3.7.0 (scala: 2.13) - Deployment: broker + zookeeper We suspect the symbolic link is the culprit, because if we set "log.dirs" to the `/actual/path/to/logdir` directly, there is no such problem. Interestingly, if we set "log.dirs" to one subfolder deeper, for example `/some/path/to/logdir/subdir`, it also didn't complain. I searched the internet but could not find a description saying that "log.dirs" cannot be a symbolic link, so I am asking for help. Thanks, David