---------- Forwarded message --------- 보낸사람: hello <leepeter2...@gmail.com> Date: 2021년 6월 9일 (수) 오후 2:50 Subject: Kafkabroker log swap writing error To: users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019= gmail....@kafka.apache.org <users-sc.1623217635.lobeokoidbfekkpndnbk-leepeter2019= gmail....@kafka.apache.org>
Whenever I run Kafka server in windows 10, it failed to rename controller.log to controller.log.time After several failed attempts, it stops running. *java.nio.file.FileSystemException: C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned -> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap:* *“The process cannot access, because the other process is in use”* *[2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in C:\Testing\.logskafka-logs have failed (kafka.log.LogManager* Do you have any idea how to fix this? [2021-04-20 15:47:53,282] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 9 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-20 15:47:59,474] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 9 (__consumer_offsets-31) (reason: Adding new member node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 with group instance id None) (kafka.coordinator.group.GroupCoordinator) [2021-04-20 15:52:53,286] INFO [GroupCoordinator 0]: Member node1-agent8-4e7ee271-581a-40b2-9b1f-e5fb4f364704 in group monitoragent has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator) [2021-04-20 15:52:53,287] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 10 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-20 15:52:53,295] INFO [GroupCoordinator 0]: Assignment received from leader for group monitoragent for generation 10 (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-15]. [2021-04-20 16:06:59,437] INFO [GroupCoordinator 0]: Member node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 in group monitoragent has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-15]. [2021-04-20 16:06:59,438] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 10 (__consumer_offsets-31) (reason: removing member node1-agent8-16cc2029-7ee0-4560-9890-028fea05b8b2 on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator) [2021-04-20 16:06:59,439] INFO [GroupCoordinator 0]: Group monitoragent with generation 11 is now empty (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-16]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-17]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-18]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-19]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-20]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-21]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-22]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-20-23]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-00]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-01]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-02]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-03]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-04]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-05]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-06]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-07]. [2021-04-21 08:11:56,092] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 11 (__consumer_offsets-31) (reason: Adding new member node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 with group instance id None) (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-20-16]. [2021-04-21 08:11:56,095] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 12 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:11:56,100] INFO [GroupCoordinator 0]: Assignment received from leader for group monitoragent for generation 12 (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:16:46,832] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 12 (__consumer_offsets-31) (reason: Adding new member node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e with group instance id None) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:20:22,323] INFO [GroupCoordinator 0]: Member node1-agent8-bd039ca4-dca7-43a9-bd31-48b6a9d86031 in group monitoragent has failed, removing it from the group (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:20:22,326] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 13 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:20:22,383] INFO [GroupCoordinator 0]: Assignment received from leader for group monitoragent for generation 13 (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:23:38,927] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 13 (__consumer_offsets-31) (reason: Adding new member node2-agent9-2210cbbe-9324-4d53-9b0e-a37997275127 with group instance id None) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:26:23,712] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 14 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 08:26:23,722] INFO [GroupCoordinator 0]: Assignment received from leader for group monitoragent for generation 14 (kafka.coordinator.group.GroupCoordinator) [2021-04-21 09:01:02,878] INFO [GroupCoordinator 0]: Member[ group.instance.id None, member.id node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e] in group monitoragent has left, removing it from the group (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-08]. [2021-04-21 09:01:02,880] INFO [GroupCoordinator 0]: Preparing to rebalance group monitoragent in state PreparingRebalance with old generation 14 (__consumer_offsets-31) (reason: removing member node1-agent8-4a3f8f6f-1091-4954-99c1-d1ff2c8ae12e on LeaveGroup) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 09:02:27,729] INFO [GroupCoordinator 0]: Stabilized group monitoragent generation 15 (__consumer_offsets-31) (kafka.coordinator.group.GroupCoordinator) [2021-04-21 09:02:27,786] INFO [GroupCoordinator 0]: Assignment received from leader for group monitoragent for generation 15 (kafka.coordinator.group.GroupCoordinator) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-08]. log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/controller.log] to [C:\kafka_2.13-2.7.0/logs/controller.log.2021-04-21-09]. [2021-04-21 10:27:53,405] INFO [ProducerStateManager partition=__consumer_offsets-31] Writing producer snapshot at offset 70374 (kafka.log.ProducerStateManager) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/server.log] to [C:\kafka_2.13-2.7.0/logs/server.log.2021-04-21-09]. [2021-04-21 10:27:53,412] INFO [Log partition=__consumer_offsets-31, dir=C:\Testing\.logskafka-logs] Rolled new log segment at offset 70374 in 11 ms. (kafka.log.Log) log4j:ERROR Failed to rename [C:\kafka_2.13-2.7.0/logs/log-cleaner.log] to [C:\kafka_2.13-2.7.0/logs/log-cleaner.log.2021-04-20-15]. [2021-04-21 10:27:59,102] ERROR Failed to clean up log for __consumer_offsets-31 in dir C:\Testing\.logskafka-logs due to IOException (kafka.server.LogDirFailureChannel) java.nio.file.FileSystemException: C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.cleaned -> C:\Testing\.logskafka-logs\__consumer_offsets-31\00000000000000000000.index.swap: “The process cannot access, because the other process is in use” at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileCopy.move(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source) at java.nio.file.Files.move(Unknown Source) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:917) at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:211) at kafka.log.LazyIndex$IndexValue.renameTo(LazyIndex.scala:155) at kafka.log.LazyIndex.$anonfun$renameTo$1(LazyIndex.scala:79) at kafka.log.LazyIndex.renameTo(LazyIndex.scala:79) at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:496) at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2338) at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2338) at scala.collection.immutable.List.foreach(List.scala:333) at kafka.log.Log.replaceSegments(Log.scala:2338) at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:614) at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:539) at kafka.log.Cleaner.doClean(LogCleaner.scala:538) nsumer_offsets-31\00000000000000000000.index.swap: “The process cannot access, because the other process is in use” at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) at sun.nio.fs.WindowsFileCopy.move(Unknown Source) at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source) at java.nio.file.Files.move(Unknown Source) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:914) ... 18 more [2021-04-21 10:27:59,120] WARN [ReplicaManager broker=0] Stopping serving replicas in dir C:\Testing\.logskafka-logs (kafka.server.ReplicaManager) [2021-04-21 10:27:59,124] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35, __consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1, __consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12, fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21, __consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7, __consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2, fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25, __consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49, fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30, fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16, fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13, __consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42, __consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31, fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24, fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23, __consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45, __consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19, __consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8, __consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2, __consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38, __consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6, fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37, fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5, __consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28, fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1, __consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29, __consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32, fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15) (kafka.server.ReplicaFetcherManager) [2021-04-21 10:27:59,128] INFO [ReplicaAlterLogDirsManager on broker 0] Removed fetcher for partitions HashSet(fuzz_seed-36, __consumer_offsets-35, __consumer_offsets-37, fuzz_seed-42, fuzz_seed-34, fuzz_seed-1, __consumer_offsets-22, fuzz_seed-17, __consumer_offsets-30, fuzz_seed-12, fuzz_seed-47, fuzz_seed-27, __consumer_offsets-8, __consumer_offsets-21, __consumer_offsets-4, __consumer_offsets-27, __consumer_offsets-7, __consumer_offsets-9, __consumer_offsets-46, fuzz_seed-33, fuzz_seed-2, fuzz_seed-18, fuzz_seed-0, fuzz_seed-43, __consumer_offsets-25, __consumer_offsets-41, __consumer_offsets-23, __consumer_offsets-49, fuzz_seed-26, fuzz_seed-19, __consumer_offsets-33, fuzz_seed-30, fuzz_seed-39, fuzz_seed-14, __consumer_offsets-47, __consumer_offsets-16, fuzz_seed-3, fuzz_seed-48, __consumer_offsets-28, fuzz_seed-13, __consumer_offsets-31, __consumer_offsets-36, __consumer_offsets-42, __consumer_offsets-3, __consumer_offsets-18, fuzz_seed-46, fuzz_seed-31, fuzz_seed-9, fuzz_seed-35, __consumer_offsets-15, __consumer_offsets-24, fuzz_seed-29, fuzz_seed-4, fuzz_seed-20, fuzz_seed-23, __consumer_offsets-38, __consumer_offsets-48, fuzz_seed-45, __consumer_offsets-17, fuzz_seed-49, fuzz_seed-40, __consumer_offsets-19, __consumer_offsets-11, fuzz_seed-21, fuzz_seed-16, fuzz_seed-8, __consumer_offsets-13, fuzz_seed-10, __consumer_offsets-2, __consumer_offsets-43, __consumer_offsets-6, fuzz_seed-38, __consumer_offsets-14, fuzz_seed-25, fuzz_seed-22, fuzz_seed-6, fuzz_seed-41, __consumer_offsets-20, __consumer_offsets-0, fuzz_seed-37, fuzz_seed-32, fuzz_seed-11, __consumer_offsets-44, fuzz_seed-5, __consumer_offsets-39, fuzz_seed-44, __consumer_offsets-12, fuzz_seed-28, fuzz_seed-24, __consumer_offsets-45, __consumer_offsets-1, __consumer_offsets-5, __consumer_offsets-26, __consumer_offsets-29, __consumer_offsets-34, __consumer_offsets-10, __consumer_offsets-32, fuzz_seed-7, __consumer_offsets-40, fuzz_seed-15) (kafka.server.ReplicaAlterLogDirsManager) [2021-04-21 10:27:59,220] WARN [ReplicaManager broker=0] Broker 0 stopped fetcher for partitions fuzz_seed-36,__consumer_offsets-35,__consumer_offsets-37,fuzz_seed-42,fuzz_seed-34,fuzz_seed-1,__consumer_offsets-22,fuzz_seed-17,__consumer_offsets-30,fuzz_seed-12,fuzz_seed-47,fuzz_seed-27,__consumer_offsets-8,__consumer_offsets-21,__consumer_offsets-4,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,__consumer_offsets-46,fuzz_seed-33,fuzz_seed-2,fuzz_seed-18,fuzz_seed-0,fuzz_seed-43,__consumer_offsets-25,__consumer_offsets-41,__consumer_offsets-23,__consumer_offsets-49,fuzz_seed-26,fuzz_seed-19,__consumer_offsets-33,fuzz_seed-30,fuzz_seed-39,fuzz_seed-14,__consumer_offsets-47,__consumer_offsets-16,fuzz_seed-3,fuzz_seed-48,__consumer_offsets-28,fuzz_seed-13,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,__consumer_offsets-18,fuzz_seed-46,fuzz_seed-31,fuzz_seed-9,fuzz_seed-35,__consumer_offsets-15,__consumer_offsets-24,fuzz_seed-29,fuzz_seed-4,fuzz_seed-20,fuzz_seed-23,__consumer_offsets-38,__consumer_offsets-48,fuzz_seed-45,__consumer_offsets-17,fuzz_seed-49,fuzz_seed-40,__consumer_offsets-19,__consumer_offsets-11,fuzz_seed-21,fuzz_seed-16,fuzz_seed-8,__consumer_offsets-13,fuzz_seed-10,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,fuzz_seed-38,__consumer_offsets-14,fuzz_seed-25,fuzz_seed-22,fuzz_seed-6,fuzz_seed-41,__consumer_offsets-20,__consumer_offsets-0,fuzz_seed-37,fuzz_seed-32,fuzz_seed-11,__consumer_offsets-44,fuzz_seed-5,__consumer_offsets-39,fuzz_seed-44,__consumer_offsets-12,fuzz_seed-28,fuzz_seed-24,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,__consumer_offsets-34,__consumer_offsets-10,__consumer_offsets-32,fuzz_seed-7,__consumer_offsets-40,fuzz_seed-15 and stopped moving logs for partitions because they are in the failed log directory C:\Testing\.logskafka-logs. (kafka.server.ReplicaManager) [2021-04-21 10:27:59,224] WARN Stopping serving logs in dir C:\Testing\.logskafka-logs (kafka.log.LogManager) [2021-04-21 10:27:59,232] ERROR Shutdown broker because all log dirs in C:\Testing\.logskafka-logs have failed (kafka.log.LogManager)