On Tue, Nov 10, 2020 at 3:09 AM Ivan Kelly <iv...@apache.org> wrote: > Hi folks, > > It's been about a year since Streamlio joined Splunk and since then > we've had a bit of forking with our BK branch. > It has gotten to a stage where it's starting to be a problem for us, > so we'd like to start to get things back in sync. > > There are a couple of big chunks of work to come back. > We've added a data integrity checker that replaces a lot of the > functionality of autorecovery and allows us to run without a journal. > We refactored the bookie to allow dependency injection. > We've rewritten the entry logger to use direct I/O (allowing 2GBps > writes per bookie). >
+1 Looking forward to the changes. > > One other thing we've done is to change the build system to use gradle. > The major driver for this was that maven is just slow, even before you > start running tests. > "mvn clean package -DskipTests" takes 4m30 on my laptop. "./gradlew > clean jar" takes 40s. > Subsequent builds on gradle are much much faster, as it does > incremental building. > Incremental building exists in maven, but it doesn't work. > Gradle also handle multimodule projects better. If I make a change in > bookkeeper-common, > "./gradlew :bookkeeper-server:test" will pick up the change without > having to explicitly > "mvn install" the bookkeeper-common. In my opinion it's just a much > nicer build system > to work with. Even the poms it generates are better as they avoid > dependency pollution. > > What are peoples opinions on moving BookKeeper to gradle (assuming > I/splunk do the legwork)? > If people are open to it, I'll submit a BP. > +1. My only question is how do you do an Apache release. I'd like to see BP covering that question. > > Another thing that BK (and the whole ecosystem) is missing is > structured logging. > We also plan to add structured logging to BK in soon. This is a major > motivator for converging the branches, > as it touches a lot of places. > > Anyhow, any feedback appreciated. > > -Ivan >
