Krishna Sai Dandu created KAFKA-19438:
-----------------------------------------
Summary: Request for Assistance: FileSystemException Errors on
Kafka Log Cleanup in Windows (KRaft and Zookeeper Modes)
Key: KAFKA-19438
URL: https://issues.apache.org/jira/browse/KAFKA-19438
Project: Kafka
Issue Type: Bug
Components: config
Affects Versions: 3.3.0
Reporter: Krishna Sai Dandu
Fix For: 3.3.0
Hi Kafka Dev Team,
We are encountering critical issues while running Apache Kafka in our
application and would appreciate your assistance.
h3. Application Context:
* We are using a *Spring Boot Java application* that depends on {*}Apache
Kafka{*}.
* The application and Kafka brokers are running on a {*}Windows system{*}.
* We have tested with:
** *Kafka 2.8.0* (Zookeeper mode)
** *Kafka 3.3.0* (KRaft mode – Zookeeper-less)
h3. Observed Issues:
h4. *Error 1: During Log Cleanup in Consumer Offsets*
Kafka attempts to perform log cleanup when the log file size reaches 100 MB,
but encounters a file lock error:
java.nio.file.FileSystemException:
C:\tmp\kraft-combined-logs-1__consumer_offsets-21\00000000000000000000.timeindex.cleaned
->
C:\tmp\kraft-combined-logs-1__consumer_offsets-21\00000000000000000000.timeindex.swap:
The process cannot access the file because it is being used by another process
This results in the broker shutting down gracefully but unexpectedly.
h4. *Error 2: During Topic Log Retention Deletion*
When a topic log file reaches the retention threshold, a similar error occurs:
java.nio.file.FileSystemException:
C:\tmp\kraft-combined-logs-1\Persistance-1-0\00000000000000000000.index
->
C:\tmp\kraft-combined-logs-1\Persistance-1-0\00000000000000000000.index.deleted:
The process cannot access the file because it is being used by another process
This again causes the broker to shut down.
*server.properties Configuration Snapshot:*
process.roles=broker,controller
node.id=1
controller.quorum.voters=1@localhost:9093
listeners=PLAINTEXT://localhost:9092,CONTROLLER://localhost:9093
inter.broker.listener.name=PLAINTEXT
advertised.listeners=PLAINTEXT://localhost:9092
controller.listener.names=CONTROLLER
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
log.dirs=/tmp/kraft-combined-logs-1
num.partitions=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
h3. Request:
We would like to understand:
* Is this an issue related to {*}Kafka configuration on Windows{*}?
* Is this {*}expected behavior due to Windows file locking mechanisms{*}?
* Are there any *recommended workarounds or configurations* for stable Kafka
operation on Windows environments?
This issue is blocking our production deployment. Your guidance or any leads
would be greatly appreciated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
