Hi, A bit late to this discussion, there are few emails in the thread I'll respond to separately.
Yes it's just to implement a binary file based queue that supports realtime tailing. If it had been low friction it would have made more sense, but given the friction Chronicle's approach to OSS brings I don't think it makes much sense to keep using it. It has some jankiness to it. I vaguely recall having copied the files while they are being written to and then gotten errors trying to read them. I think the next time we want to add functionality to the FQL/binlog it would be worth re-implementing that part or even just doing it to get rid of the dependency. Ariel On Mon, Sep 16, 2024, at 5:16 PM, Benedict wrote: > > Don’t we essentially just use it as a file format for storing a couple of > kinds of append-only data? > > I was never entirely clear on the value it brought to the project. > > >> On 16 Sep 2024, at 22:11, Jordan West <jw...@apache.org> wrote: >> >> Thanks for the sleuthing Stefan! This definitely is a bit unfortunate. It >> sounds like a replacement is not really practical so I'll ignore that option >> for now, until a viable alternative is proposed. I am -1 on us writing our >> own without strong, strong justification -- primarily because I think the >> likelihood is we introduce more bugs before getting to something stable. >> >> Regarding the remaining options, mostly some thoughts: >> >> - it would be nice to have some specific evidence of other projects using >> the EA versions and what their developers have said about it. >> - it sounds like if we go with the EA route, the onus to test for >> correctness / compatibility increases. They do test but anything marked >> "early access" I think deserves more scrutiny from the C* community before >> release. That could come in the form of more tests (or showing that we >> already have good coverage of where its used). >> - i assume each time we upgrade we would pick the most recently released EA >> version >> >> Jordan >> >> >> On Mon, Sep 16, 2024 at 1:46 PM Štefan Miklošovič <smikloso...@apache.org> >> wrote: >>> We are using a library called Chronicle Queue (1) and its dependencies and >>> we ship them in the distribution tarball. >>> >>> The version we use in 5.0 / trunk as I write this is 2.23.36. If you look >>> closely here (2), there is one more release like this, 2.23.37 and after >>> that all these releases have "ea" in their name. >>> >>> "ea" stands for "early access". The project has changed the versioning / >>> development model in such a way that "ea" releases act, more or less, as >>> glorified snapshots which are indeed released to Maven Central but the >>> "regular" releases are not there. The reason behind this is that "regular" >>> releases are published only for customers who pay to the company behind >>> this project and they offer commercial support for that. >>> >>> "regular" releases are meant to get all the bug fixes after "ea" is >>> published and they are official stable releases. On the other hand "ea" >>> releases are the ones where the development happens and every now and then, >>> once the developers think that it is time to cut new 2.x, they just publish >>> that privately. >>> >>> I was investigating how this all works here (3) and while they said that, I >>> quote (4): >>> >>> "In my experience this is consumed by a large number of open source >>> projects reliably (for our other artifacts too). This development/ea branch >>> still goes through an extensive test suite prior to release. Releases from >>> this branch will contain the latest features and bug fixes." >>> >>> I am not completely sure if we are OK with this. For the record, Mick is >>> not overly comfortable with that and Brandon would prefer to just replace >>> it / get rid of this dependency (comments / reasons / discussion from (5) >>> to the end) >>> >>> The question is if we are OK with how things are and if we are then what >>> are the rules when upgrading the version of this project in Cassandra in >>> the context of "ea" versions they publish. >>> >>> If we are not OK with this, then the question is what we are going to >>> replace it with. >>> >>> If we are going to replace it, I very briefly took a look and there is >>> practically nothing out there which would hit all the buttons for us. >>> Chronicle is just perfect for this job and I am not a fan of rewriting this >>> at all. >>> >>> I would like to have this resolved because there is CEP-12 I plan to >>> deliver and I hit this and I do not want to base that work on something we >>> might eventually abandon. There are some ideas for CEP-12 how to bypass >>> this without using Chronicle but I would like to firstly hear your opinion. >>> >>> Regards >>> >>> (1) https://github.com/OpenHFT/Chronicle-Queue >>> (2) https://repo1.maven.org/maven2/net/openhft/chronicle-core/ >>> (3) https://github.com/OpenHFT/Chronicle-Core/issues/668 >>> (4) >>> https://github.com/OpenHFT/Chronicle-Core/issues/668#issuecomment-2322038676 >>> (5) >>> https://issues.apache.org/jira/browse/CASSANDRA-18712?focusedCommentId=17878254&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17878254
