kafka:ERROR Shutdown broker because all log dirs(Windows)

发布时间 2024-01-09 10:43:22作者: eiSouthBoy

问题

都说kafka在Windows上运行的错误多,果然如此。在测试阶段,创建了一个 测试主题,并往该主题发送了消息,然后删除该主题。这时错误出来了:
kafka error log日志

[2024-01-08 15:22:42,224] ERROR Error while renaming dir for test-topic-0 in log dir C:\Users\admin\kafka\tmp\kafka-logs (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: C:\Users\admin\kafka\tmp\kafka-logs\test_topic-0 -> C:\Users\admin\kafka\tmp\kafka-logs\test-topic-0.7ab06cc1e97e4f76bd05476086dc7d09-delete
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:395)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:292)
	at java.base/java.nio.file.Files.move(Files.java:1421)
	at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:935)
	at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:918)
	at kafka.log.LocalLog.$anonfun$renameDir$2(LocalLog.scala:111)
	at kafka.log.LocalLog.renameDir(LocalLog.scala:786)
	at kafka.log.UnifiedLog.$anonfun$renameDir$2(UnifiedLog.scala:698)
	at kafka.log.UnifiedLog.renameDir(UnifiedLog.scala:1752)
	at kafka.log.LogManager.asyncDelete(LogManager.scala:1135)
	at kafka.log.LogManager.$anonfun$asyncDelete$3(LogManager.scala:1170)
	at scala.Option.foreach(Option.scala:407)
	at kafka.log.LogManager.$anonfun$asyncDelete$2$adapted(LogManager.scala:1168)
	at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
	at kafka.log.LogManager.asyncDelete(LogManager.scala:1166)
	at kafka.server.ReplicaManager.stopPartitions(ReplicaManager.scala:470)
	at kafka.server.ReplicaManager.stopReplicas(ReplicaManager.scala:407)
	at kafka.server.KafkaApis.handleStopReplicaRequest(KafkaApis.scala:298)
	at kafka.server.KafkaApis.handle(KafkaApis.scala:180)
	at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:75)
	at java.base/java.lang.Thread.run(Thread.java:834)
	Suppressed: java.nio.file.AccessDeniedException: C:\Users\admin\kafka\tmp\kafka-logs\test-topic-0 -> C:\Users\admin\kafka\tmp\kafka-logs\test-topic-0.7ab06cc1e97e4f76bd05476086dc7d09-delete
		at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
		at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
		at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:309)
		at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:292)
		at java.base/java.nio.file.Files.move(Files.java:1421)
		at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:932)
		... 17 more

zookeeper error log日志:

[2024-01-08 15:22:42,770] WARN Close of session 0x10029db71750001 (org.apache.zookeeper.server.NIOServerCnxn)
java.io.IOException: 远程主机强迫关闭了一个现有的连接。

这时候再次启动kafka,发现再也启动不了。网上说把 tmp 文件夹全部删除,就能启动。确实可行,但这样kafka所有的数据也就丢失了,很闹心。

解决(无解)

问题:ERROR Error while renaming dir for test-topic-0 是由于 rename操作 引起,网上查询结果是,由于 log4j.jar 包中的 renameTo() 方法在Windows上有问题,但是在Linux上没有问题。
有人通过修改 log4j.jar 中的源码 链接,从而避免在Windows出现renameTo()方法的错误。

至于有没有一种方法能不修改 log4j.jar 源码能解决上述问题,至少目前还没有查到,如果有这种方法后续补充吧。