It is a deliberate change in JDK code Just for reference see this discussion on nio-dev list on OpenJDK http://mail.openjdk.java.net/pipermail/nio-dev/2018-April/005008.html
see https://bugs.openjdk.java.net/browse/JDK-8168628 Cheers Enrico 2018-03-05 14:29 GMT+01:00 Enrico Olivelli <eolive...@gmail.com>: > Workaround: > as these brokers are only for test environments I have set very small > values for index file size, which affects pre-allocation > segment.index.bytes=65536 > log.index.size.max.bytes=65536 > > If anyone has some thought it will be very appreciated > Cheers > > Enrico > > > 2018-03-05 13:21 GMT+01:00 Enrico Olivelli <eolive...@gmail.com>: > >> The only fact I have found is that with Java8 Kafka is creating "SPARSE" >> files and with Java9 this is not true anymore >> >> Enrico >> >> 2018-03-05 12:44 GMT+01:00 Enrico Olivelli <eolive...@gmail.com>: >> >>> Hi, >>> This is a very strage case. I have a Kafka broker (part of a cluster of >>> 3 brokers) which cannot start upgrading Java from Oracle JDK8 to Oracle JDK >>> 9.0.4. >>> >>> There are a lot of .index and .timeindex files taking 10MB, they are for >>> empty partiions. >>> >>> Running with Java 9 the server seems to rebuild these files and each >>> file takes "really" 10MB. >>> The sum of all the files (calculated using du -sh) is 22GB and the >>> broker crashes during startup, disk becomes full and no log more is >>> written. (I can send an extraction of the logs, but the tell only about >>> 'rebuilding index', the same as on Java 8) >>> >>> Reverting the same broker to Java 8 and removing the index files, the >>> broker rebuilds such files, each files take 10MB, but the full sum of sizes >>> (calculated using du -sh) is 38 MB ! >>> >>> I am running this broker on CentosOS 7 on EXT4 FS. >>> >>> I have upgraded the broker to latest and greatest Kafka 1.0.0 (from >>> 0.10.2) without any success. >>> >>> All of the other testing clusters on CentOS7 (same SO settings) did not >>> have any problem. >>> >>> The broker is given 4GB or RAM and 4G of Max Direct Memory Size (on >>> Java8 it works fine with 1GB and default max direct memory size) >>> >>> Which are the relevant configuration options ? >>> >>> Thoughts ? >>> >>> Thank you >>> >>> Enrico >>> >>> >>> >>> >> >
